This type of situation needs to be managed. As difficult as it is to make changes in your life and your mother’s life you;’ll need to face the situation now, directly. If the psychiatrist warned the family and told you to put him out, I’d honor that advice. I would encourage you and your husband to begin a detailed discussion with your mom about what is best for all of you. The goal is to keep the dialogue going with her so you can keep helping her understand your concerns. Also, I’d be certain that your mother’s affairs are in order. It sounds like you may not be able to trust your brother and will have to deal with him even after your mom is gone.
Also, as a side note, research shows that successful aging comes for doing more things for yourself — not by being waited on hand and foot.