Yes, a broad interpretation of the Commerce Clause can have a downside, and does have a downside to those who view it in the strictest sense.
IMO, it has probably run it's course. I believe the last major USSC case which cited the Commerce Clause was struck down; United States v Lopez restored some sanity to the Commerce Clause, and the opinion in the case makes for good reading.
Might want to have a look at US v Morrison.