If you're looking for inexpensive, but decent quality yarns, try Knitpicks. If you don't have something specific in mind to knit already, they have some sampler packs so you can try their various yarns in a small project.
There are a number of popular inexpensive brands available at places like JoAnn or Michael's or Hobby Lobby, and that sort of place...those can be a great place to start, especially if you're on a budget. Then you might try finding a local yarn store to go through and fondle a wide variety of yarns, usually of the more expensive and higher quality kind. I found it helpful to check out the yarn by feeling it and reading the labels to check gauge and such when I was just beginning.
I also bought up a bunch of yarn on sale that, after a time, I realized I wasn't going to be likely to use up because my tastes changed over the year or two before I got around to using it!
To keep from being overwhelmed, I'd choose a pattern or two,and buy some appropriate yarn for the project you have in mind. That will give you an idea how certain yarns feel and how they knit up. Then, after you finish those projects, go on and try something different.
Hope that helps. Jan is totally right about everyone liking different things. I heard so much about Noro, that I was thrilled about taking a class for a project that called for it---but honestly, I hated it, and will probably never buy it again...so you just can never tell!
I'm Knittermom9 on Ravelry