If that ever happened to me, I'd move the fence. It ain't my dad-gum land, and if I accidentally put a fence on it, I'm wrong!
I'm talking about a case where the fence was there when both parties moved in.
It did happen to me. Neighbor got a survey due to a dispute with his neighbor on the other side. The survey moved all of the property lines for six lots down, five feet towards me. (A thoughtful person would've wondered if there was a fault with the survey) Idiot neighbor takes this survey to the county and has it recorded. Neighbor he's having the dispute with sees that he now has five more feet of property and he takes it. Like, with a backhoe, clears it and builds a fence. Stupid neighbor spends the next ten years arguing with my grandfather about the "new" location of the property line. Never gets anywhere.
When I'm in the process to buy the property from grandparent's estate, stupid neighbor files a lien against the property to foul the title. My attorney successfully convinces their attorney that we can show adverse possession, regardless of any survey, but at any rate the survey had since been shown to be in error. Followed up with a threat to proceed on a charge of slander of title if we couldn't agree both on the "old" property line PLUS a three foot easement for maintenance. So he lost. Then he went back and tried to get the other neighbor to give back what he'd basically been given, and he was told to pound sand... after that ten years, is was his through adverse possession.
So now his property is five feet narrower, and as far as I'm concerned he deserves it.