It sounds like you may have given your boyfriend too many chances. It makes sense to me that you would remain skeptical about his recovery, and his fidelity.
I think if you are going to go forward you may want to get the opinion of a good couples counselor. Use the find help tab at the top of this page to help you locate someone in your area with that specialty.
To be honest, your boyfriend sounds like he has a long way to go. The fact that he gets angry at you illustrates that he may not understand how his behavior affects others.
I would strongly recommend you see a couples counselors sooner than later. If your boyfriend refuses to go, you may want to consider going for yourself to develop support as you decide how –- or if — you want to proceed with the relationship.