No such right has been taken away. This is about taxation, not free speech.
"Churches should look after people's souls and leave politics alone": THAT IS ABSURD!
Absurd how? I guess my idea that churches should look after souls is pretty radical after all.
It's using the threat of taxation to repress (some) churches' right to free speech, which is a violation of the First Amendment by virtue of prohibiting the free exercise of religion, as well as by virtue of giving preferential treatment to religious groups that are not subject to taxation despite engaging in political speech regularly (as with the example of Riverside Church). For background on the historical context of this topic, a source you might find an interesting read is William J. Federer's The Original 13: A Documentary History of Religion in America's First Thirteen States.