I’m not detecting much love in this relationship. In fact, I hear of systematic alienation, a desire to be separate from him, resentment, unfairness, and a lack of support. My question is more fundamental than you’re working status: Do you want to remain together? Would you be better off on your own? Do you have a reason to stay together beyond financial?
The fact that you don’t even want to take a vacation with him speaks volumes. Even if you had time to be together it doesn’t sound like you want to around him. I think I’d ask these other questions first as the retirement issue may be secondary.