When I swatch I usually cast on enough to make about a 5 or 6 inch square so you can measure somewhere in the middle of the swatch.
When a ball says "20 sts= 4inches" you are getting 5 stitches per inch (20sts divided by 4 inches). For your pattern, it says "4 stitches times 7 rows" which means you should get 4 stitches per inch and 7 rows per inch.
Have you seen those handy little guage checker thingies? Mine has a cut out shaped like an L with marks for inches, etc. and it helps you count your stitches left to right and your rows up and down. I've found that row guage isn't sooo critical at least in the projects I do.. My first baby blanket seemed too short row wise and I'm guessing it had to be a pattern typo because I knit it once thinking "I don't need no silly swatch!" and did it over again, my stitch count was right on the money so I just added rows until it measured correctly. My biggest concern is stitch gauge, that can make or break the width of lots of things.. like fitted items etc.
What seems confusing too, is that your pattern doesn't tell you how many rows but gives stitch size per the single inch, and the yarn ball gives you the stats in a 4 inch size. I usually see on yarn balls that the standard is 4 inches/10cm so you want your swatch to be slightly bigger to play around with.
Sooo, since it seems the yarn calls for more stitches with smaller needles. my best guess would be you should be able to try it on 5 mm (size 8 ) . It seems your pattern calls for yarn just one stitch away from the recommenced amount on the yarn. I have a pattern from Scarf Style that specifies a certain Debbie bliss yarn with size 7's but the yarn ball says 8-11...chances are, with my tension, I'll use the patterns recommendation instead of the yarn ball's specs.
I hope I didn't confuddle you even more... honestly, I think doin' the math part is waaaayyyy harder than actually knitting!!!
(oops, had to edit... i read your needle as US size not metric and suggested 5's to mean size 5 not 5 mm.. fixed now)