I ,too, haven't knit too much that requires a swatch. I'm in the midst of my first sweater, for which I did a regular swatch...we'll see if I did it right!
At one point, I thought I was doing the raglan decreases wrong, and wanted to fix the issue...so I picked up a new ball and swatched that part of it...and found I was right after all...glad I did that and didn't frog first!!
Otherwise, I do mostly hats, socks and scarves, if the hats/socks don't fit, I can give them to someone else. I have two pairs of socks that are slightly looser than I like, and one pair tha is cast on a little too snugly...but I've figured out yarns and needles that work together for me to get the right fit...as long as I stick with similar sizes of yarn, I'm good to go.
I figure for patterns and drape, I can get started on the project and if I don't like it I can frog and it's no worse than having swatched...but if I do like it, I've saved time.
If I decide I enjoy the sweater sorts of projects, I see much more swatching in my future!