You are right to be concerned. I doubt there is a person reading this who would encourage you not to worry. His addictive behavior isn’t likely to come to a halt without some serious effort on his part — and for all the nice things he does and buys there isn’t much he is doing in terms of self-control for the future. Sex addiction, if that is what you think is going on, is one of the more difficult behaviors to get under control. “Wanna screw?” And then dismissing your concerns with: “You’re not my wife,” pretty much tells you how little your feelings matter.
I wouldn’t be so concerned about his past as I would be about going forward. If he promises to have sex or flirt with someone else whenever the two of you are having a fight — this isn’t such a great sword of Damocles for the relationship to live under. Any fight has the potential to lead to his betrayal. Are you willing to be in a relationship where, at the first sign of difficulty, he may find it easier to screw someone else than to work it through?
Most people wouldn’t want to keep dealing with it. You’ll have to decide if it is worth it.