Why! I'm sure their was some provision in the church bylaws for getting rid of a pastor. Why didn't they ride him out of town on a rail and get a pastor that wasn't living in sin?
You can darn well bet if this happened in MY church it wouldn't be the congregation that left...
Either the poofter runs a vary authoritarian operation where he has handpicked those that would be able to throw him out and thus nothing happened, or the place was basically a personality worshiping outfit and folks moved on to a new personality. You see this a lot in independent fundamental baptist circles.