You can, but it’s counter to the image projected. Only experienced fans know it doesn’t have to be a “formal” evening out. Which is why it would be good to advertise some shows specifically as “casual”. Changing images is what advertising is for.
It’s mostly because of how it’s depicted in films and tv shows. Whenever you see a shot of the audience it’s like some alternate universe where everyone is a patrician in formal wear and the monocle has remained in style. If you look at actual audiences at the opera or symphony it’s mostly people who look and dress like students, school teachers and secretaries.