IRS is very important agency for existence of a functioning government, without which we would descend into anarchy. IRS has the tough job of collecting legal tax due.
That said, IRS can not be allowed to operate as a ruthless dictator who can freeze or seize assets of tax payers without a court hearing.
Strange we functioned fine from 1787 to 1916 without a irs.