See and I have never had an issue with it being smaller.
My Long tail cast on looks and stretches the same as the rest of my kniting. But I've been doing that cast on exclusively since for ever. And I have to say, it wasn't always that way.
There is no rule, is there!
I'm doing a Patons Cable pattern right now and it did make a difference as to counting the first row. When I have the time I'll take a pic of my WIP and show!