Let's clarify a bit the situation and what you can do about it, ok?
First, he has the right to be where he is. The only thing you have to consider here are his legal rights.
Yes! He can live where he lives now and no, he doesn't have to move away from you if he doesn't want to.
So what to do about it?
Here are two key ideas you have to strengthen in you:
- Respect and diplomacy!
These are the two key qualities you have to remember at all times when interacting with him. No fights or arguments. No "us" conversations. No opening up of your feelings to him.
This already gives you clear boundaries on how to relate to him.
- What he does is none of your business
This gives you the second guideline you need to stick to at all times.
He is free! He can see whoever he wants whenever he wants.
This is what you have to repeat to yourself.
Pain arises when you try to control something that is no longer yours. This is where lots of tension arises.
Practice these two key attitudes for a week and make sure you start interacting with other men as well.
Having someone new in your life is the final answer in your situation, so start walking towards that direction as soon as you can.