It appears that your boyfriend has what is called a benign paraphilia. It’s “benign” because it’s essentially harmless. For some reason, he is aroused by the feel and sight of these pants and socks. Where it may cross the line to something more risky is his desire for the two of you to go out in public in them. If he is aroused by the activity, he may inadvertently offend the neighbors – which could lead to trouble for him.
It doesn’t matter if his fetish is or isn’t common. I’m not sure if there is a way to know. What matters is that you are uncomfortable. Part of becoming a couple is figuring out how to have a rich intimate life that is satisfying for both of you. That means talking frankly about what is erotic for each of you and finding ways to accommodate each other. You say you don’t want your sex life to revolve around wearing the pants but are you willing to play with it a little? Can the two of you find a way to integrate his desires with your own? Talking this through is likely the next step in discovering each other and deciding if you are a couple.
I wish you well.