You could get a pastoral council whose documents are merely advisory and do not bear the weight of infallible pronouncenments, and hence are not protected by the Holy Ghost from error. This is what Vatican II was.
Worse still, you could get a 'robber council' like Ephesus.
By their nature, all Ecumenical Councils are infallible. Gainsaying a tautology is neither an arguement nor an explanation.
I respectfully suggest you do some more research on "Infallibility." #891 of the Catechism of the Catholic Church states "The infallibility promised to the Church is also present in the body of bishops, when, together with Peter's successor, they exercise the supreme Magisterium, above all in an Ecumenical Council."
If you accept the Catechism as valid, then I don't see how you can assert that Vatican II was not protected by the Holy Ghost.