No, you’re not completely off. From what you say, this mom is not only “babying” her youngest son, but is toying with seeing him as a sex object as well. You can’t change their relationship. He can — but only if he wants to. And he may not want to. From his point of view, he has life pretty easy. All he has to do is sacrifice his own adulthood, and he isn’t expected to do anything. He quite literally isn’t “driving” his own life.
There are equally pressing things to think about (besides his relationship with his mother) as you decide whether to pursue a romance with this man. Why would you want to hitch your life and your future to someone who has never finished school and has never had a job? Unless something shifts remarkably, he will not leave his mother for you. Even if by some miracle he did, he would probably expect you to pick up where mom leaves off — supporting him and babying him.
I encourage you to take a big step back. You deserve to have a man in your life, not an overgrown baby. However sweet he is, eventually you will want him to do his share in supporting the two of you and making a life together. Give him the opportunity to “man up” and see what he does. Make your decisions from there.
I wish you well.