I remember watching a KG show one time, and the guest (maybe Lily Chin??) suggested an easy way to remember. Start with crochet needle size "H." "H" sounds like "8," so correspond the "H" to size 8 knitting needle. As you go down through the alphabet, the knitting needle numbers get smaller, and they go up as you go higher in the alphabet.
Look at this
site. The chart is very helpful.