Three people (landowners) use the easement. No one has been affected by the fence in the last 15 years. It simply runs alongside the road which is on the easement.
If your fence is within that easement, it was likely put there illegally. The fence has to be totally on your property and NOT within the bounds of the easement. Have had this situation before. The fence was ordered removed. Because no one complained doesn't make it right.
It sounds like he wants to improve the drive and that's part of the agreement...shared costs of the party.
I think you might be leaving pertinent stuff out.
Why does your neighbor NEED a gate for the fence? Does the fence prevent him from coming on to YOUR PROPERTY or is there a common path, etc. involved?