Anybody who responds so vehemently can’t have your best interest at heart. This is important. Don’t let yourself be swayed by his reaction. Explain that you’re bothered enough that you are leaving the bed. This isn’t your problem alone — both of you are involved in. While the issue is of a different caliber, countless couples go through this with one accusing the other of snoring. The offender is often unaware and demands evidence. Your husband may not know he is doing it, he may be lying, but in either case if you are being woken regularly — it needs to be resolved. Explain that a sexless relationship and calling you a crazy person isn’t the life you are looking for. On the other side of the coin, the next time you are woken up by this reach over and find out exactly what’s happening.
Whatever is happening I think couple’s therapy would be a good idea. The find help tab at the top of the page can help you find a therapist in your area.