Sounds like you've done it perfectly. Now you want to:
Rows 3&4 : work to 4 sts past neck opening, w&t.
So knit across 17sts with the first ball of yarn, drop that strand, pick up the strand from the second ball and knit 4sts. That'll take you 4sts past the neck opening, w&t. Again, like the back it sounds like you'll be working one side for a few rows.