The answer to your question isn’t simple. Although some survivors find it helpful to confront their abuser, that isn’t always the case. Sometimes it results in retraumatization. Sometimes the abuser threatens further abuse.
The best person to discuss this with is the therapist who has already helped you so much. He or she knows the situation and probably has an opinion about whether it would be dangerous for you to try to contact the abuser for a confrontation.
I am concerned that you continue to be triggered so much by seeing the place where the abuse took place. It might be helpful to do some more work with your therapist. An alternative is for you to find a way to move from the area so you can move on. Yes, I know. It isn’t fair for you to have to move. He should. But sometimes reality demands that the survivor take care of herself by getting out of the environment and away from the abuser in order to move on. Do talk to your therapist about it.
You have already done a great deal of positive, good work to heal. I suggest that the next step is to discuss your question with your therapist. Therapy often takes a number of episodes. Do take this your letter and this response with you as a way to get into the discussion.
I wish you well.