Size 3 needles are sort of large for standard fingering weight sock yarns (unless you have a VERY tight gauge.) I'd be looking for DK weights if you want to stick with that needle size.
As for the fluffies, sure, you could, but durability is always a question when using non-wool/non-reinforced yarns for an item that will be, well, worn.
I'm not sure how comfortable a fluffy yarn would be on the soles, either, but it depends what you mean by "fluffy". Maybe use the fluffy for instep and cuff?