Sorry I was being confusing; when I started slipping the stitches purlwise it helped but didn't completely fix the problem. I still have wrong color showing up and I'm not sure which rows are causing me the problems.
My guess is that it's the even rows that are causing me the issue. From the pattern:
"Row 2 and all alternating rows: Use the same color from previous row to knit all knit stitches and slip all slipped stitches with the yarn in front."
Is there something more to knitting all knits and slipping all slips that I'm missing? I'm knitting all the knit stitches and all slipping all the slipped stitches purlwise on the even rows.