Actually, what matters is your tension.
The pattern will say something like "Tension: X sts and Y rows over 10cm". Let's pretend it says 20 stitches and 26 rows, just to have some numbers to work with.
Knit up a square, about 12cm X 12cm and then measure a square 10cm X 10cm within that. (You knit the extra 2 cm to give yourself a border round your actual measured square for more accuracy.) Count how many stitches you have across (left to right) and how many rows you have (up and down). If it matches the pattern requirements, then you're fine. If you got too many stitches (across), say 22, then try bigger needles. If you got too few, say 18, then try smaller needles. The same with the rows-- too many, try bigger needles, too few, try smaller needles. However, the stitches across are more important than the rows up and down. Sometimes when switching yarns, you'll be able to get one to match up and not the other. In that case, as I said, make sure the stitches across are exact. You may have to knit several squares, but it is well worth it. The tension square is something a lot of people are tempted to ignore, and then they are very sorry when they have an unwearable jumper after all of that work. (That 1 stitch more or less really adds up in the entire way round the jumper!)
Now, you CAN do things like adjusting sizes to compensate, but it's more complicated, especially if your jumper has set-in sleeves with a fitted sleeve cap, versus a boxy construction with sleeves just attached. For now, since this is your first one, I'd go with the matched tension.