did you try pinning it first? and then sewing it together... I have a hard time sometimes lining up the sides without first sticking them in the correct position.
Also, what kind of stitch are you using for your seam? I use a "whipstitch" and the seams generally look pretty good.