Also, if possible, it would be best to add the new color in on a knit row if possible (and if your pattern allows for it). It can be added in on a purl row, but the joining in technique is a little trickier, I find.
Are you using a specific pattern that calls for a color change? And if so, does it say where you're supposed to add in the new color? If you are using a pattern (vs. just doing some practice knitting), it might be helpful if you posted the part where the color change takes place.