You are right.
One major hurdle in eliminating the IRS is that it is currently a prime tool for legislators to reward or punish their favorite groups. As a result, it’s a major source of (bribery) income to legislators. It’s a prime reason that so many lobbyists are making big bucks. And it’s a tool used to terrorize political enemies. Congresscritters will be reluctant to give up such a powerful tool of income and terror.
Far too much power is located in DC... The Uni-party will fight to keep it. The 17th has to be repealed or nothing will be lasting, no true reform can occur. The 17th Amendment is like a form of Cancer on our Republic. It has to be eliminated in order for the patient to recover and survive.
Imagine 50 or is it 57 State Governors, involved and up for elections on different cycles... even with current technology, the power in DC would checked to an appropriate level.