Maybe he didn’t want the man on his property.
He lost that right when he agreed to living in an HOA. Were this normal property, I'd be right with you, and him. He lost in court, too.