Nah, that's the way it works.
Most people audition for lower level producers, who are not particularly rude or helpfully constructive...and then they go home.
The producers specifically pick the best and near best to show the judges plus the most colorful, weirdest and worst.
The TV show is staged as if EVERYONE sings for the judges, and they are being "bombarded" by awful singers, when actually those people are being hand-picked because of their degree of self-delusion, or pity postcard stories, plus, of course, the real hopefuls who have gotten a chance to sing for the judges.
American Idol goes out of its way to NOT spotlight most of the singers who wind up in Hollywood.
I also heard a quote from Simon that went something like: I always knew there were a lot of great singers in America, and this year proved it. He said this year was the best talent collection for the show they have seen.