About 4 is the youngest I've managed to teach so far. Younger than that, they'll sit and watch, sometimes grab at the needles and make flailing attempts, but the patience to listen isn't there yet. (I've also had a lot of stuff torn up by toddlers trying to "help" and parents who think it's cute to watch their kids tear the stitches off the needles, throw the yarn around, etc.) A truly interested toddler can try finger knitting with a little big yarn, but I watch them like a hawk to make sure they aren't putting the yarn around their own necks or anyone else's.
Pure anecdata: my son used to sit on my lap while
I knitted or crocheted, and when he finally asked to try to knit (and I thought it was safe to let him) he could. He didn't find it all that interesting, so he doesn't, but he could if he had to. He had learned enough from watching to get the idea.