Thank you for asking this question. I think it is something that many others try to cope with.
There are two ways to understand this. Both have value. One way is to say that you’ve been cheated on in the past and to believe that you bring this past experience with you into the current relationship and it taints it. The second is that you may be attracted to men who are not faithful, and you are confirming your suspicions. I am throwing my vote with the second.
This doesn’t mean you should end your relationship. But it does mean you need to be VERY clear with your boyfriend about what is acceptable and what is not. You may want to look for an individual therapist or couples therapist to work this through. You can find some recommendations at the find help tab at the top of the page.
His excuse is lame. You have a right to be upset, and ask him to stop. Your self-deprecating remark about your stupidity over being insecure indicates you are too willing to take the heat for his behavior. It is important for you to speak up and be clear about what isn’t okay. Him saying “it doesn’t mean anything” completely disregards the fact that it did mean something to you. It is important for you to honor your feelings about this and ask him to be responsive to your wishes. Otherwise, if he cheats on you he will simply say the same thing. I am encouraging you to deal with it sooner rather than later. If he can’t, won’t or gives you a hard time about it, move on. You are looking for a man who can honor your needs.