Yep, exactly. You need to shape the sleeve to accommodate both your skinny l'il wrist and your muscular upper-arm comfortably.
That's why you increase more often down by your wrist, and less often up above the elbow.
Congrats on re-discovering knitting! It's like riding a bicycle - you never quite forget how, once you learn. Relearning is fun.
When you're done, it'd be great if you could post a photo of your sweater masterpiece!