As a 46-year-old man, you certainly don’t have to put up with this. You don’t need to convince your father of anything. You know what happened to you. It doesn’t matter what he thinks. Your truth matters, not his.
You ask if you should confront or walk away. I suggest to you that there are more alternatives. There is no point in arguing with him. He isn’t likely to change. Walking away sends the message that what he is doing is okay with you – which it isn’t. It also leaves you feeling ashamed.
I think it would be more helpful if you could make a clear statement to your dad of how it makes you feel and what you will do any time he approaches you inappropriately.
You could say something like, “Dad. It is entirely inappropriate for you to touch me like that or say those things to me. You are my father, not a boyfriend. Fathers don’t treat their sons that way. If you persist, I will only see you when other people are around. I’m not interested in being around you when you treat me with such disrespect.”
Then follow through. The very first time he says or does something that makes you uncomfortable, leave. Don’t talk about it, shout about it, argue about it, or repeat yourself. Shouting and talking and arguing send the message that perhaps you can be talked out of your opinion. Taking silent and sure action sends a far more powerful message to him and preserves your dignity as an adult man.
It’s very sad that your father’s behavior puts a wedge between the two of you. I understand that you have wanted him to be different and hoped that he would be the kind of loving, supportive dad you’ve always wanted. But you can’t make him change. All you can do is hold on to your own self-respect by setting a clear boundary and sticking to it.
I wish you well.