I know exactly what you are talking about.
And I've been in your shoes many times!
If the pattern says that row gauge is equally critical, then you must trust that information.
The only thing you can do at this point is try other needle sizes. You didn't say if your row gauge is too tall, or too short...but use a size up or size down depending.
If you can't find a needle size that will achieve the required stitch & row gauge with the yarn you are using...then you will either have to sack the pattern, or, find a different yarn that WILL conform to gauge.
You won't be the first knitter that has had to give up on a pattern using an unsuitable yarn. I've had to face it many times.
I want to commend you for taking the time and care to check your gauge!
It's the most important "beforehand work" you'll ever do when knitting garments!
The Knitters' Three-Step!