Can we really be sure that he won’t do it again, and open himself up once again to blackmail? His word means nothing, based on him breaking the biggest promise he made in his life. Why take the chance?
In the Bible, David did far worse and yet he was called a man after God's own heart. If he had a history of numerous affairs, that would be a different matter.