It is never okay for an adult to be sexual with a child. The definition of molestation is relative because people use it to mean different things. Some people consider molestation to mean sex. Others define it as inappropriate touching or unwanted sexual advances.
Your father may have touched you in a sexual way. If he did it was wrong and inappropriate. It may constitute sexual abuse.
You recognize that you and he have a strained relationship. Your memories may explain why you feel uncomfortable around him. The problem is that you may never know the full truth.
Should you tell your brother? The answer is “it depends.”
What is the motivation behind telling your brother? How might he react? Would it anger him? Those are questions to consider, among others. If you and he are close and you feel this knowledge would benefit him in some way, then you may want to tell him. If you believe it will cause problems or be confusing, then you may not want to reveal it to him. It is difficult to give you a specific answer because I am not familiar with the family dynamics or the nature of the relationship between you and your brother.
If there are children in the family, for instance if your brother has children, then this situation should be discussed. It should begin with your talking to your father. After that if you are not satisfied with the results of the discussion, you should tell your brother.
If this is an issue that is bothering you then you may want to consult a therapist. You can give him or her more details and he or she may be able to give you a more specific answer. I am sorry that I could not give you a definitive answer but there are many variables to consider regarding whether or not you should reveal your memories to your brother.
Here is link that may help you find a therapist in your community. Thanks for your question.