I appreciate the courage in asking the question publicly. What arouses people sexually can be along a very wide spectrum. The fact that there is something to turn you on is not a bad thing. Best of all situations can be when couples learn to turn each other on and use these activators as a way to stimulate, enhance, and evolve the relationship. So while I know you said that this isn’t something you feel you can share with your wife, if there is ever an opportunity or a possibility that you could—that would be the best. Then it moves the fetish from something that feels like it has to be kept a secret into something that can be used for reciprocal pleasure.
On the other side of the coin is knowing how to use this activation with your wife in a more subtle way. The natural version of this, if she is willing, would be for her to keep her panties on during foreplay as a source of stimulation. My point is don’t turn this into something bad if there is a way it can be used to enhance the relationship sexually. Of course if it becomes too uncomfortable for you and there isn’t a way it can be incorporated you can always ask the help of a therapist in learning to manage the impulses.