When you bring the yarn forward between the needles for a purl you need to remember to move it back for the knit stitches. If you leave it forward it creates what is called a 'yarn over'. Yarn overs are valid stitches and they create a little hole. They are often used in lace, for increases, or as button holes.
It just takes practice to remember to move the yarn back.