Are you using basic stockinette or garter stitch?
Where are you changing yarns/colors? end of row? middle of row?
Are you using straight needles or a cable needle?
A photo would help but you likley just put the work down and when you picked it back up, started going the wrong way (i.e. had the needles in the wrong hands).