I'm sorry you had a bad experience.
Felting itself is not hard, but I wonder about the "fancy yarn" you used. Was it some sort of non felting yarn that you mixed with the yarn that was to felt, or was it an animal fiber (was it mixed with something else, or the only yarn you knit with for that part? If it was some really hairy, but not so good yarn, maybe the hairys couldn't hold up to the rigors of the felting process and .....fell out. That could spell disaster. I'm not sure if you can do anything to make it look like it did before felting if a bunch of the fibers fell out of the fabric.
Go ahead and cry a little, it might help you get over it and feel better.