As long as it's only a couple of rows it should be fine. Twisting the unused color when you come around to the beginning may keep any small holes from forming and helps carry it up the side.
I've done fair isle socks and my hint to you... work loosely or you won't get them on your feet!
