Interesting. I've made a bunch of hats (over 30) to the same overall pattern, made in joined rounds, not a spiral.
The seam is discernible, but not that visible/intrusive/objectionable. Is it possible that you're unconsciously tightening the stitches at the beginning or ending of the rounds, thereby making the join more of a "feature" than it's meant to be?
And, no, I haven't kept all 30 hats! I have maybe seven, but one only sees the light of day at Christmas, and a couple of the others are for sleeping (we do our best NOT to turn on the heat or to keep it as low as we can). Most of the hats have gone to a Chemo Hat program; now they're going to another service program. That way, it doesn't matter so much that they're on the same pattern.