Another recommendation I've come across: Knit the sleeves first as gauge is less critical to fit there and if you work from the cuff you can measure the actual knitting as you go up. FWIW....or not. I've also seen: Knit the hat first.
You're not crazy, you're a knitter.
There's a cure for crazy.