John Locke’s opposition to the divine right of kings was an aspect of this issue. The divine right of kings was not a medieval doctrine, though it did go back to oriental despotism, to the divinization of Alexander the Great and the Roman emperors. Authority came directly to the king, not through the people, as the Aristotelian mind had it. Divine right was designed to protect the king from assassination by elevating him to a divine status.
IMO the core issue is that of delegated vs absolute authority. If authority is delegated, then it can be rescinded by the grantor. If authority is conferred and rendered absolute, then it cannot be revoked. Protestants and Western Civilization leans towards the former. Catholicism leans toward the latter with regards to popes, bishops, and priests.
Yes, I see how that could be a thorny issue. If you believe the Holy Spirit is involved in the safeguarding the selection process, then the authority was not only conferred, but conferred with the imprimatur of God. How then, could a mistake have been made, and one who would turn heretical have been appointed? The only route to revoke the authority, it would seem, would be to argue that the appointment was not done properly, and was null from the beginning. Basically, you would need to argue for an “annulment”, not a “divorce”.