I like the buttermilk, periwinkle and lime together too, but the primary colors are a good idea also. I never thought about these bright colors for baby items but I will in the future, I love bright colors!
I am knitting with crayon right now and haven't had a single problem with it. I have also used homespun and to me they don't seem similar at all, but I don't have many problems with homespun either. The crayon doesn't have all the loose loops and frizzy-ness of homespun and I haven't gotten my needles stuck in it once. Not sure if that helps or not since each person's preference can be so different. That is just my take with those two yarns.