No. You’re not wrong. Part of growing up and making a family is transferring primary allegiance from one’s family of origin to the new family. One of the responsibilities of the older generation is to support the new family in doing this. Ideally, there is room for everyone to have good relationships with everyone else while putting emphasis on creating a safe and secure environment for the children. It sounds to me like your husband hasn’t found a way to become a loving husband first and a respectful son second. He has been raised to think that doing so risks his membership in his original family. Of even greater concern is that the so-called adults are modeling for their children that drugs, partying, and hurting others are appropriate ways for adults to behave.
Fighting only exhausts you and makes your husband’s family gang up on you all the more. My suggestion is that you drop the fight and do all you can to develop your career and a life outside the scope of this family. If it gets to the point that you can’t stand it any more, you’ll have the means to separate and a group of friends who can offer you support. Meanwhile, protect your children from the situation as best you can. For the sake of peace, you might want to make an appearance at family events but I see no reason for you to stay very long when family members can’t be civil to you and can’t be reasonable role models for your children.
I wish you well.