I'm no big spindle expert but I did try spinning with one a couple of years ago and noticed the same thing as you just did. I just picked it up again and tried after successfully learning how to spin on a wheel. Much better this time. The difference was not paying attention to what the spindle was doing, just watching the twist in the wool itself. In other words, even if the spindle is slowing or even spinning back, the wool itself may be twisting up the right way because of the twist already in the single.
Also, the fluffing you refer to is called predrafting. Try doing that and you will have much more success. There are some great videos on You tube on how to do this.