If the “keys” were intended for all then why was it that Jesus only gave them to Peter, and not to the other apostles. Jesus used very similar language in addressing the other apostles in Matthew 18, and gave them the power to “bind and loose”, but they did not get the keys. Why would they not receive the keys too? Why was it only Peter?
I cannot see how one can read Matthew 16:18-20 and conclude that the keys were for all.
Taking that one verse and building an entire doctrine on one verse is nonsense. Scripture tells us 2 or 3 witnesses. That applies to scripture as well when we want to determine what is meant. If our view of one verse does not fit with other portions of scripture we must continue to search for the meaning.
Jesus tells us that wherever two or three are gathered in His name He is among us.
Matthew 18:18 Verily I say unto you, Whatsoever ye shall bind on earth shall be bound in heaven: and whatsoever ye shall loose on earth shall be loosed in heaven. 19 Again I say unto you, That if two of you shall agree on earth as touching any thing that they shall ask, it shall be done for them of my Father which is in heaven. 20 For where two or three are gathered together in my name, there am I in the midst of them.
That promise was not just to the Apostles but to all believers who trust in Him.
John 16:23 And in that day ye shall ask me nothing. Verily, verily, I say unto you, Whatsoever ye shall ask the Father in my name, he will give it you.
Even back in Malachi 3 we see that God honors even just two who gather in His name.
Malachi 3:16 Then those who feared the Lord talked with each other, and the Lord listened and heard. A scroll of remembrance was written in his presence concerning those who feared the Lord and honored his name.
No large organized conglomerate. No long established hierarchical organization. Just 2 or 3 individuals gathered in Jesus name have all the power of God available to them if they ask according to the will of the Father. He wishes for no one to be lost but we are told to first go to an individual who is doing wrong by ourselves and if he won’t listen to us then to take 2 or 3 but if he still won’t listen we are told then, and only then to take the matter to the “church” which we know is the local gathering of believers and address the issue. If he still at that point refuses to listen we are to have nothing to do with that individual. That’s a directive to all the believers of that local “church” not just to the elders of the church.