"This sort of logic means that no Pope has the authority to excommunicate anyone as a schismatic, since schism depends on the interior state of a person."
Not true. A papal tribunal in which charges are made and subjects are duty-bound to respond and to defend themselves usually had been the proper forum in the past for high churchmen. The modernists were too clever for this, though. Archbishop Lefebvre had the whole of Catholic Tradition on his side. It would have been counter-productive to openly charge him with schism in a tribunal in which he might properly defend himself and prove beyond a shadow of a doubt that he was innocent. So Lefebvre's enemies--which included the Pope--decided on the automatic excommunication, by first giving him the runaround on the business of the consecrations until he had no other choice but disobedience or surrender to the new doctrines.
They argue that the sentence of schism and excommunication pronounced against them by the Archbishop of Tyana, the Apostolic Delegate in Constantinople, was unjust, and consequently void of strength and influence ...Certainly, if you recall the history of your districts, you will find examples of Roman Pontiffs who used this power when they judged it necessary for the safety of the Eastern Churches. This was why the Roman Pontiff Agapetus used his authority to eject Anthimus from the See of Constantinople and replace him with Mennas without calling a synod. (Bl. Pius IX, Quartus Supra)
I don't see anything about tribunals here.
by first giving him the runaround on the business of the consecrations until he had no other choice but disobedience.
But in fact there was no way of coming to an agreement. While I was facing Cardinal Ratzinger with that alternative, and while he was saying that he would give us a bishop on the 15th of August, he was asking me for still more dossiers in order that the Holy See might choose a bishop who would meet the requirements laid down by the Vatican. Now, where was that going to lead us?
or surrender to the new doctrines
Since these "new doctrines" are non-existent (I presume you mean Lefebvre's distorted conception of Dignitatis Humanae?), there wouldn't have been much of a surrender.