No no, knitting is not a waste of time!
I do understand your frustration, though.
My sweater attempts have been much like yours, and I don't understand what the problem is. The gauge swatch is sometimes misleading because I swear my knitting tension changes when I move from the swatch to the actual project.
Maybe for that adorable pattern you've posted, you should just go down a needle size or two and see if that helps. Just don't give up!!
You might treat yourself to a quick, fun project, just to get your mind off the sweater woes and to make knitting fun again for you.