I don't have the chart but from the gauge, that sounds right and yes it sounds small. Usually the neck seems too large but the collar (and especially a turtleneck) closes up the space.
Jan's advice to start the sweater is good. You won't have to knit very far to see if this will work. Do use a very stretchy cast on.
Here are two projects from Ravelry. You might try sending a message to both of the knitters who have recently finished the sweater and to Ann Budd about the neck.