I get both thumbs, both index fingers and my middle finger black from gold, but of course I'm using a polishing machine to achieve that.
It could be the acid in or on your skin, especially if it's a lower karat gold, but I would lean towards something on your skin polishing the metal.
Are you using hand lotion or could you be getting some make up that has talc in it on your fingers?
People who "jinx" watches is why I never got into watchmaking even though I have the training and was very good at it.
I pick up a watch and it works without doing anything to it, couple that with someone who picks it up and breaks it and it would be an impossible job.