You are correct. You are drowning in self-pity and you have a long history of substituting blame for action. You were never “pushed” into someone else’s arms. The fling with the colleague was a decision you made to make yourself feel better. It’s not “unfortunate” your wife found out. It’s the usual outcome of such poor decisions.
This is not to say that you haven’t had a troubled marriage for a very long time. It appears that you and your wife never figured out how to be on the same team. You still haven’t. Children have to be fed and cared for. A house has to be kept up. Finances have to be dealt with. Instead of negotiating how to accomplish these tasks together, you or your wife have each taken on tasks and then resented the other person. It’s not a way to make a marriage. Sadly, you have apparently never gone for counseling to learn how to have a cooperative and mutually satisfying life.
It’s not too late to save your marriage, unless you two decide it is. You loved each other enough to marry and to bring children into the world. With some hard work you can reclaim that love and learn how to have a mature relationship.
Even if you don’t do that, you do need to do some individual work to learn how to be a functioning partner in a functioning relationship. Otherwise, you’ll bring the same lack of problem-solving skills to whatever new relationship you decide to pursue. However much you may think you are in love with the married woman, a relationship with her is bound to fail, if you don’t take responsibility for your part in the failure of your marriage and learn how to forge a different kind of relationship. That’s necessary whether you decide to make another go at your marriage or if you are to be successful as a divorced co-parent of your children or if you decide to start over with someone new.
I wish you well.