Sometimes gauge swatches lie. I'm not the only one who's found that my gauge changes part way through a project. It happens. Not for everyone maybe, but it does happen. I think newer knitters are more likely to not notice it until a project is completed because of getting used to knitting and holding the yarn and learning the feel of things. Experience teaches us to check now and then to see if gauge and other measurements are still what they should be. As time's gone by I've noticed that I can tell if the yarn is moving more or less easily so my chances of having different gauges and not noticing for a long time are lessened. Experience is the best, and maybe the hardest, teacher.
You're not crazy, you're a knitter.
There's a cure for crazy.