I have to use IE6 at work, and there really is a difference.
In Firefox, when you put your mouse over "My Notebook" the little menu drops down. Not in IE.
When browsing the forums, there's a bar at the bottom of the screen; in Firefox, it stays where you can see it at all times and easily click to reply, move to the next topic, mark read, whatever. In IE, its at the very bottom so you have to scroll all the way down to see it.
Things just aren't nearly as snappy in IE as they are in Firefox. Pages/images take longer to load, etc.
Go with Firefox.