If you knit the first/last stitch of the row (instead of slipping it like many do), are you making sure that the bump is on the back (or the front when you turn)? Does that make any sense? I had problems with that when I started. I'd turn that first stitch into two because I had both the loops from the bump under it and knit those. Just make sure that your purl bump is in front and you are using the last stitch not the purl bumps' yarn.