The real question here isn’t if it is normal, it is if you want to be in a relationship where sexual intimacy isn’t exclusive. Here are the questions that I think are in front of you: What are your goals in a relationship? Will you be resentful of him for this exploration? Do you feel this puts you at risk? What are your needs in this or any relationship?
I think these are very deep questions and my advice is to not push away your discomfort, but to examine it and decide if this relationship gives you what you need now, and whether it has the potential to do that in the future.