When you switch colors, you have to hold the one you're stopping over to the left and bring the new one up from under it. This will twist the stitches and avoid the hole. Do it on both sides.
Check out Amy's video on intarsia if you're confused.
As for the holes that are still there, just stitch them closed.
"Trust the pattern!"
Time flies like an arrow, but fruit flies like a banana.
Hop aboard while we follow the cloud-covered van of knitting harmony, man!! Woooo!! --Hedgehog
IngridKH on Ravelry