sweater frustration

This new sweater project is really frustrating me. I started the first front side 4 times before I felt the gauge & size were right. Then things seemed to be going along fine & I knit the other front side & started on the back. I then decided maybe I should measure the front pieces & make sure they were the right width. The second one was at least 2 centimeters too skinny. I guess my gauge was too tight though I did keep checking it, so I ripped it out. Then I checked the gauge on the back & it was too small so i took it off the needles but then when I measured it it was the right width so I put it back on the needles. Maybe I need to just measure my gauge over a longer section to be sure. Not sure why I am having such a hard time with this sweater…knitting food is so much easier, it doesn’t need to fit!

Oh yeah & its got only two rows of garter stitch at the edge so its rolling badly & its not supposed to, I don’t think blocking is going to stop it.