I sort of hijacked Artlady's thread with my story but in short it was someone I know who said she coudn't knit because it was 'doing nothing'. Mostly my reaction was disbelief... because, really, you probably shoudn't say it to people who have less productive hobbies or even just sit on the couch. Maybe they are coming up with a new groundbreaking idea or praying for world's peace?
Anyway, recently I knit a lot in public cause I have tiny 'on the go' projects and keep the yarn in my pocket. I use dpns for those so some people stare. Mostly they don't even comment... maybe because it's hard to make eye contact or something. And usually I am surrounded by parents who are busy whatching their kids or playing with their phones while waiting. If someone asks if it's hard, I show a few stitches slowly but that's about it.
I probably use knitting in public not only to fill the time but also as a bate for other knitters since I have no knitting friends... It hasn't worked yet but maybe if I sit outside in a high traffic area and smile to everyone, it will