Maybe you should try weaving every other stitch like they do @ Philosopher's wool? I really think that floats longer than 5 sts are pretty challenging/ annoying.
That slubby KP Crayon I was using was the worst, too!
I kept getting irritated because it's only a FEW ROWS of colorwork! How could it be that bad? But I have had so much better luck using wool! My pregnant friend is vegan so I couldn't use wool for the hat.
Then again, I guess my current colorwork might be too tight, too. I didn't think so, though. I'm going to block those "bracelets" over a wine bottle and I bet it will be ok. For the shoulder areas I have started with one size larger needles. That's size 7!!! I used size 4 for the hems and size 6 for the bottom of the sleeves. (The pattern called for size 4 and 5).