Yep. In order for the knitting to lie flat, you should change colors frequently. You could, for example, strand the teeth, but for any solid block, you need to change strands. If you try to strand across the back on some parts, they tend to look thicker than the rest--because that section IS two strands thick.
a lot of ends to weave in, but if you weave them in along the edge of the color change, it's not too much of a pain.
Just think of what a cool sweater that would be!