If you're sure you have picked up all the dropped stitches, then I think you should be fine. I bet if you gently tug at the fabric a bit, the stitches will settle back into shape. However, if you think it's going to bug you, I'd frog now while it's only 3-4 rows rather than waiting till it's significantly more.
Mommy to two wonderful girls
"I don't have to conform to the vagaries of time and space, I'm a loony!" -Campbell Bain (David Tennant) 'Takin' Over the Asylum'
OTN: Cabled Sweater
, arm warmers, kids' clothes