It sounds to me like you are feeling caught between two men you love. You worry about your son but you also want to back your husband. In times like that, it’s good to look for some help.
The reason this situation isn’t getting resolved is that no one is facing the real problem. Your son is wasting his time and your money. Your husband is angry with him but isn’t dealing with the issue directly. His anger then comes out indirectly with put downs and the order to stay home. Your part in this is that you are more worried about your husband’s behavior than your son’s. A family discussion is long overdue.
The topic that you are all avoiding is that your son may not be ready for more education. He certainly isn’t taking advantage of the enormous gift you and your husband are giving him. Rather than call off a vacation, maybe it’s time to call off college.
Your son needs to take some time to figure out what he wants to do with his life and whether he wants and needs college to get there. He could get a job and an apartment and experience the working world for awhile. Another option is to spend a year volunteering with a program like CityYear or Habitat for Humanity or any of scores of organizations that are desperate for help. Doing something worthwhile for others and dealing with supporting himself will probably do wonders for his self-esteem and will open up new possibilities.
There is no need to be angry when talking with your son about this. This isn’t about punishing him. It’s about all three of you facing the reality that school may not be for him. He’s a young man now. It’s time all three of you expected him to take responsibility for himself.
I wish you well.