Worsted weight on size 6 needles will give you a pretty dense fabric, which is probably what you'd want for a coffee cozy.
To determine how many you need for your yarn, knit a swatch with needles that give you a fabric that you like. You'll probably want to go smaller than 6. Since the cup won't complain about the size
you probably could get away with casting on about 20 stitches and knit for an inch or so. That will give you an idea of how many stitches per inch you're going to be knitting at.
Figure out how many inches around your cozy should be (a little smaller so you have a bit of room for stretch) and multiply that by the number of stitches per inch you get.
So, for example, if your cozy is 5 inches around, and you get 5 stitches per inch, you'd cast on 25 stitches.