It seems to me that your husband might have unconsciously left the pipe where you could find it in order to open up a conversation. There’s a difference between confronting someone and expressing concern. I hope you can do the latter. Talking about whether he has broken a promise or lied to you will only get in the way of what really needs to be talked about: the fact that there is a problem in your relationship that needs to be named and dealt with.
My first guess is that both of you are overwhelmed. You two had over 10 years together before you had children. Now, here you are at 40 with two children who need all your time and attention and most of your funds and your love. It can be exhausting. Many couples aren’t prepared for what a big shift this is. It could be that you “escape” by thinking about whether you still love your husband and fantasizing about what it would be like without him. It could be he “escapes” by getting stoned.
The trouble with this strategy is that you may “escape” yourselves out of a perfectly fine marriage. If I’m right about this, you would do better to try to work on the problem directly. Find times to appreciate and show love for each other throughout the week. Set up a date night for just of the two of you. It doesn’t have to be elaborate and expensive. A coffee in a quiet cafe or a walk around the neighborhood will do if you really talk. Can’t afford or find a sitter? Arrange with other parents to swap babysitting chores so they can get out for a night too. If this doesn’t work please consider seeing a couples therapist to help you through this tough spot.
Kids don’t stay young and needy forever. Faster than you can imagine, they will be asking for the car keys and leaving you for long evenings alone together.
I wish you well.