This is happening because it is your first scarf! Everybody's first scarf does that.
If what you are doing is what I did with my first scarf: You may not be pushing both strands off when you knit some of your stitches, or you might be inadvertently catching a strand from the row below when you do push it off (so you have one new stitch and one mysterious new stitch).
The solution (the one that worked for me):
Use stitch markers (coloured paper clips work) to count how many stitches you have each row (it makes it easier to double check the number of stitches and can show you if you are creating new stitches in the middle of the row or at the ends). When you are on the first or last stitch of each row, fiddle with the end stitches to make sure it hasn't twisted. Try to be very careful with the last stitch (that's where mine tended to increase).