You are not crazy. Your boyfriend is simply not owning up to his behavior. The lying, secrecy, and compulsive gambling problem are actually rather telltale signs. Let me be blunt. If he is willing to gamble your savings and stick you with a tax burden, be emotionally engaged with a woman via texting, and not tell you about contacting his ex, then what is next? The lying, cheating and stealing are small now, but consistent. Him telling you it is your problem and you are crazy is standard fare. He doesn’t want to change, so blaming you is the solution.
Hold your ground. Ask him to go to couples counseling with you. I hope he does, but if not, please find a therapist to start working with so you can start thinking about your options. These kinds of things tend to get worse without couples therapy. The find help key at the top of the page will help. To answer your original question: You can’t trust him because he isn’t trustworthy.