nope shorter will be fine...that's typical as you start decreasing...in some cases might even have to go down to dpns (but probably not for a sweater!

) you are going to mess with the sweater more if you are trying to force too few stitches over a needle that is too long than the opposite.