I usually find mine at thrift stores, or Goodwill stores, or Salvation Army stores, or consignment stores.
The first time I went searching, it took FOREVER, because I was having to read the label of absolutely EVERY sweater I found. I've gotten pretty good at recognizing different fibers by feel, so it's much easier now. I still sometimes mistake some acrylics though!
I look for wool, cashmere, angora, and silk.
Be sure to look in the men's section also, because those sweaters are sometimes very large, and you'll get LOTS of yarn!