Please forgive that this is going to sound like I'm talking to a 6 year old-- I'm just going through every possible step in order to figure out what's going on:
Let's say you're picking up sts along the neck edge of a sweater, in order to knit the neckband. With the right side (public side) of the sweater facing you, with the needle in your left hand (pretend it's a straight needle for the sake of argument), the pointy end pointing to your right, you pick up the sts. Now, with the right side/public side of the fabric still facing you, take the other needle in your right hand, attach the yarn and start purling, the sts being transfered from the needle in your left hand to the needles in your right hand. This will make the purl bumps come out on the right side/public side of the fabric. Or am I missing something?. . .