The argument is that the ecumenical, papal affirmed body of cardinals deciding who is the valid pope is not an infallible decision, even though if it was one defining true doctrine it would be. Yet this is another issue in which RCs debate. For instance, in "Dogmatic Fact: The One Doctrine that Proves Francis Is Pope." Robert Siscoe argues,
Fr. Smith went on to explain that because the Church is an indefectible visible society, it can never adhere to a false head. There’s no need to study canon law, or spend years researching ancient Latin texts buried away in archives, to be absolutely certain that a particular pope was (or is) the true pope. All that is required to ascertain his legitimacy is to find out if he was recognized as pope by the Church. If the answer is yes, that alone provides infallible certitude of his legitimacy, as well a corresponding degree of certitude that all the conditions required for him to have become popes were satisfied — such as the condition that the papal office was vacant at the time. And the certitude of the pope’s legitimacy occurs the moment the entire Church learns of his election, provided it is not at once contested.
A dogmatic fact is one that has not been revealed, yet is so intimately connected with a doctrine of faith that without certain knowledge of the fact there can be no certain knowledge of the doctrine. For example, was the Vatican Council truly ecumenical? Was Pius IX a legitimate pope? Was the election of Pius XI valid? Such questions must be decided with certainty before decrees issued by any council or pope can be accepted as infallibly true or binding on the Church. It is evident, then, that the Church must be infallible in judging of such facts, and since the Church is infallible in believing as well as in teaching, it follows that the practically unanimous consent of the bishops and faithful in accepting a council as ecumenical, or a Roman Pontiff as legitimately elected, gives absolute and infallible certainty of the fact.
If the Church did not have infallible certitude about the legitimacy of the current and past popes, she could never be certain that a particular doctrine had been defined, or the definitive decrees of a council ratified, by a true pope or an antipope. Consequently, the object of the Faith (what Catholics must believe by faith) would be uncertain, which the devil would easily exploit to undermine the faith. The scrupulous would be paralyzed by fear, and the unstable would fall into the most outrageous conclusions. Those who denied various dogmas would only have to cast doubt upon the popes who defined them in order to justify their incredulity. This shows why the Church must have infallible certitude about the legitimacy of those she recognizes as the Roman pontiff, either past or present.
The Council of Constance formally condemned the following proposition: “If the pope is wicked, and especially if he is foreknown to damnation, then he is a devil like Judas the apostle, a thief and a son of perdition and is not the head of the holy Church Militant since he is not even a member of it.” – CONDEMNED - https://onepeterfive.com/dogmatic-fact-francis-pope (excerpts)
The argument is that the ecumenical, papal affirmed body of cardinals deciding who is the valid pope is not an infallible decision, even though if it was one defining true doctrine it would be.
The College of Cardinals does not have the charism of infallibility. The Pope acting on his own can (the "Extraordinary Papal magisterium"), and an Ecumenical Council can. (An Ecumenical Council requires all bishops in the world to at least be invited; a group of cardinals only would not qualify.)
I'm not sure what "an ecumenical, papal affirmed body of cardinals deciding who is the valid Pope is not an infallible decision" even means.
Are you referring to a Papal election? That is "infallible" in the sense that, once a validly elected Pope has validly accepted election, the College of Cardinals can't simply reverse their act "because we want to".
Perhaps you have a fundamental misunderstanding of "infallibility". An "infallible act" is not guaranteed to be perfect or ideal, merely free from doctrinal error and therefore irreversible (although perhaps subject to better explanation and broader understanding -- as I said, it's not necessarily perfect).
If you are referring to a Pope being deposed because either (a) it is determined that he was not validly elected; or (b) it is determined that he is a contumacious public heretic and has thereby relinquished his office, that would require an Ecumenical Council, not merely an agreement by a group of Cardinals. I'm still not sure what "infallibility" would mean in that context. It's an administrative act, not a doctrinal one.
I hope that defense wasn't too "venomous".