-----------------------
This is a perfect example where you two need space from each other. When you spent many years together you get used to the other's presence. This "habit" is still there. A part of you definitely wants to go.
Your fighting is an emotional response which says something like: "get off! You are too close".
On the other hand, there is this emotional attachment which sends another message and says: "I miss you and I am afraid of letting go".
The solution is to bring fresh energies into your life. Meet new people. Meet new men. Open up doors. Basically, refresh your life. Sometimes, it is easier to start anew from scratch rather than trying to clean this story.
I can imagine how this makes you feel and I know it is tough. However, this is one of the situations in which some fresh air will give you back the power and confidence you need.
This past relationship is not giving you the qualities you need. It will be easier to go and find them somewhere else rather than trying to keep on pushing a door which does not want to open.
Dare to stand alone and move forward. Get back your full freedom and full power back and open yourself to fresh opportunities.
The anger and irritation you feel is truly the sign that a part of you wants to clear itself from his presence. Anger is your emotional response to clear your mind from his presence. You want space and freedom. This is why a part of you attacks him.
You want to gain your full independence back and your anger is your way of expressing that need.
I know that what I will say might surprise you but in your situation, your anger and frustration are positive forces. They are the expression of your renewal power wanting to create space and gain total emotional freedom.