I don't think you're worried over nothing but worrying constantly doesn't do much good. It can't change the situation. Knit on! There are many other and much less productive things you could do.
This is the last time I'm frogging this project for the last time.