The thing is, it doesn't matter if you're being anal or not. The point is, they belong to you and you don't like it. Period. He should respect your desires as far as how your own tools are treated. It's a simple matter of respect.
Maybe you need to put them some place where it will be inconvenient for him to mess with them.