My first sweater didn't fit me very well either. I ended up cutting pieces out and sewing it so it would be smaller. That made it look a lot better, but it still doesn't look so great.
Look for patterns that have the size in inches instead of S, M, L, and be very careful about your gauge. Measure a sweater that you know fits you. That way you know you're knitting a sweater that is x inches around and that's the size you want.
You could read about EZ's percentage sweater system to make great sweaters without a pattern too.