A Pastor has no authority, he is given a responsibility by the Head of the Church, Jesus Christ. Authority implies control, and the ability to mete out punishment. A Pastor is simply a Shepherd keeping his flock safe and fed. Mostly with the knowledge of the Word, but also feeding the physically hungry of his flock.
Excellent point.