It could just be that the person who knit the sweater at that gauge is a loose knitter and you are a tighter knitter. As long as you like the fabric that the size 4 creates, then that's what's going to work.
Some people automatically have to go up or down 2 sizes for every pattern, so it's not an unusual thing.
"Trust the pattern!"
Time flies like an arrow, but fruit flies like a banana.
Hop aboard while we follow the cloud-covered van of knitting harmony, man!! Woooo!! --Hedgehog
IngridKH on Ravelry