You should discuss your concerns with your therapist. Be honest about your stance on religion. Honesty in therapy is paramount. You may be able to continue working with her despite her religious views. You won’t know until you discuss it with her.
Having said that, it’s possible that her advice will be filtered through the lens of religion. Given that her counseling practice takes place in a church, that should not come as a surprise.
The decision to keep a therapist should not hinge on the inconvenience of finding another one. It’s well worth your time and effort to find the right therapist. I usually recommend calling four to five therapists and speaking with them on the phone about the issues you would like help with and choosing the ones with whom you feel the most comfortable to meet in person. If you want good help, it often takes time and diligent research.
You will either be okay with a religious therapist or you will not be okay with it. Gather more information, perhaps give it more time and see how you feel and then decide your next move. If you feel a little better after each session, that is a good sign. It will take more time and more sessions to know with certainty. Good luck and please take care.