I live just outside of NYC. Whenever I go in, I always go to Knitty City. I really like this store because the people that work there are so nice and helpful. I like Yarn Co. too - and it is just a couple of streets away from Knitty City, but the store is usually crowded and it may be hard to get someone to help you.
School Products has an amazing selection of yarn - but not too much help.
The Point (in Greenwich Village) is always a fun place to check out and they have great coffee for sale.
Hope this helps....