I’m so, so sorry that you have had such a hard time. I don’t believe that having a spcial needs child is the cause of the break-down in your relationship. The cause is the fact that you and your husband were faced with a major crisis. When the individuals in a relationship and the relationship itself are strong, the couple pulls together and copes. When there is weakness, a crisis puts the problems in stark relief. As it happens, it was the baby’s needs that made those problems obvious. But it could have been anything else that required your husband to to step up and be a husband and partner.
I do suspect that he is experiencing enormous grief over the child. Rather than facing his sadness and anger that he has a baby whose needs are continuous and challenging, he is retreating to his pre-adult life; repeating the relationship he had with his mother with his daughter and acting as if his friends and mother are more central to his life than you are. Although this is in some ways understandable, it isn’t at all comfortable or helpful to you or to your relationship with each other. For now, your daughter will adore him just because he is daddy but at some point she will make more complicated demands on him too. At that point he may take more distance from her as well.
I’m very sorry he won’t consider counseling. For your relationship to survive, something needs to happen. I suggest you start counseling even if he won’t go — not because I think you are the problem but because you need and deserve some support. Your counselor can hear the whole story and may have some suggestions for getting your husband to join you in treatment. I also hope you will consider joining a parent support group for parents of kids with special needs. People often find it comforting to talk wiith other parents who are going through the same kinds of things and to exchange ideas and experiences that help your children.
I wish you well.