Small color block on circular needles. Help!

[Flappy]

I'm working on a baby hat, and I want to do something a little funky on this one. I'm using dark brown cotton yarn, and I want to put a small green rectangle in the middle of the forehead (4 stitches x 6 rows).

What's the easiest way to do this? I don't think Amy's video covers Intarsia on circular needles. I took a stab at switching to green yarn, knitting four stitches, and then knitting back to brown. Two problems with this:
o I'll end up with 24 loose ends to hide
o Four stitches isn't really enough to let the green yarn hold tension.

Any advice?