They're aggressive, for one thing, and they don't pollinate. For another, they'll wipe out a honey-bee hive in the matter of a couple of hours.
I didn't know that. I usually have YJ nests near my front door. They dive bomb visitors, but don't bother me, even if I'm shooshing one out of my kitchen. Came almost nose to nose with a nest this am when trimming clematis. I don't know why they don't bother me, but if they kill bees, I will (with some remorse) start offing them.