Ok, you left that out. That minor point, is a big part of the story.
Get a lawyer. It is pretty clear you are here looking for people to help your view of the argument. For what reason? Who knows.
Get a lawyer. Pay them a few grand. Then you won’t have enough money to build the fence. But you will have “won!”
I only left out what I felt wasn’t pertinent to the overriding question.
Can you put fence on an ingress/egress easement that in no way impedes the ingress or egress purpose?