I suppose you could adjust the pattern (or use a heavier yarn) to make it in an adult size, but WOW, you would have my intense admiration because the cast on took for-freaking-ever for the smallest baby size....I cannot even fathom having the patience to cast on 180 stitches or more in that manner!!
But that's just me....
On the floppiness issue....mine is floppier than the pattern calls for because I used a lighter weight yarn. The gauge matched but the pattern says it calls for that yarn so it would be extra dense and warm. So I think if you used maybe a worsted weight or heavy worsted weight yarn it would not be quite as floppy. I've seen a few pictures of this hat knitted in cotton and it looked much stiffer.