It would be better to use a larger needle because the yarn is a worsted and you'd normally use a size 7 or 8 with that yarn. That may get your swatch larger - if you were aiming for a 4" one and it's 1½" instead, that's way way way too small. Even if you make a bigger size, the item will be too stiff and densely knit. Using a larger needle, CO about 6 sts more than the gauge says for 4" and then don't include the edge stitches when you measure it.
I found this link
to the pattern, it even says to use a size 7. Cute hat!