If the ball band says wash and dry, I would. Did you wash and dry a test swatch? I've never had a problem with shrinkage in cotton knits, but who knows what different yarns will do. Don't dry it too long, if you put it in the dryer.
Happiness is catching that dropped stitch while it's still fixable..