What you're doing makes absolute sense, especially the part about allowing the yarn to move smoothly. Most well-written patterns have that flow.
In any event, though, I've never done a sweater where a row would make much of a difference, so there's no point in panicking.
Make sure you post a picture of it when you're done!
"Trust the pattern!"
Time flies like an arrow, but fruit flies like a banana.
Hop aboard while we follow the cloud-covered van of knitting harmony, man!! Woooo!! --Hedgehog
IngridKH on Ravelry