You and your boyfriend got together when you were only 18. At 20, you had a baby. It sounds to me like you boyfriend is ambivalent about taking on adult responsibilities and being a father. On the one hand, he probably loves his little girl and still has strong feelings for you. On the other hand, he may feel he gave up his youth too soon. If his friends are single and responsibility-free, it makes it even harder for him to move to the next stage of adult life.
The two of you need to have a very serious talk about what each of you wants out of life and your relationship. You are parents but you aren’t married. That alone tells me that both of you have had some reservations. You may not see yourself with anyone else, but you didn’t make a total commitment either.
I think you need to take a big step back from assuming you are going to be together. Come at the talk as a decision you both have to make. Are you both going to give up the single life? Are you going to marry? Are you both ready to do what it takes to make a permanent relationship with each other and a two-parent home for your daughter? If not, how are you going to co-parent? Ideally, your daughter will have both a mother and a father who are actively involved in her care and in her life. How are you going to support her financially?
You may need some help to have a productive conversation about these difficult and emotional issues. If you find you get nowhere when you try it on your own, I hope the two of you will see a couples counselor for a few sessions. The counselor’s office can give you both a safe place to say what you feel and to figure out what you want for the three of you in the future.
I wish you well.