I have a friend that knits during family gatherings like Thanksgiving. It does sound rude and anti-social.
I know it bugs my mother when my sister takes her knitting to family functions.
I can't bring myself to do it, but when I think about doing it, it doesn't feel rude. Does that make sense?
But I think non-knitters find it rude, so I don't do it.