I'm sure many people will tell you that Knitpicks.com is a GREAT place for needles and yarns at a discount. I've heard great things about their needles--though I've yet to get my own.
If you're wanting to get into knitting for the long haul...an interchangeable set might be the most economical, but it is a lot of money to drop at one time (which is what's been holding me back). Check out the stickies for people's opinions about the Boye, Denise, and Knitpicks Options sets to see what might be a good fit.
As far as basics--when I first started size 8s were probably the size I used most. Now I'm knitting more things on smaller sizes. If you want to try hats and such, a 16" circular needle is a great tool you'll get lots of mileage out of. You can knit straight items on circular needles too--so you get two types of projects out of one needle. Of if you want to give Magic Loop a go for things like hats or mittens, a 32" or 40" circular needle of whatever size will be good....and then you'll have plenty of room for stitches for things like afghans.
And when you're just starting, the cheaper the yarn the better!