Well, you already have the drafting down, which is half the battle. I think a lot of spinners who loathe the spindle initially do better when they go back after wheel spinning for a while. Try it for 2 weeks, 15 minutes a day, and see if it is enjoyable or if it just ticks you off.
It's low-risk $$$ wise, anyway -- get a good beginner spindle (schacht hi-lo, greensleeves bare bones, or cascade little si) and you're only out $20 if you end up not liking it.