I'll bet the dress is stocking stitch. Right?
If so, it is rolling up BECAUSE of the st st, not the crochet.
Maybe you could crochet an edging (other than the picot) that will stabilize the edge of the dress itself, then top it off with the picot edge.
OR, handstitch some grosgrain ribbon to the underside of the dress (like a hem)...leaving a tad bit of the edge of the dress exposed for your picot edge application.
I bagged the picot idea myself a while back and did this trim along the neckline instead of the picot edge called for:
