The first stitch on everyone's knitting is looser than the others. It's only attached to another stitch on one side.
On your next row, you can really pull that first stitch tight--almost to the point that it wraps around the needle.
Another thing you can do is to slip the first stitch of each row without knitting it. This will make for a smoother edge.
Don't rip out your knitting for a loose loop. In the long run, it all blends together and you don't even notice.
You just started knitting last week? Don't expect perfection--just keep knitting and everything will get better!