It's not just anybody's mail box. It belongs to the U.S. government, even if you bought it and installed it yourself.
My Dad was a rural mail carrier. If the cops wanted to, they could get the punks who play “mailbox baseball” in big trouble, but the cops are usually too lazy to do anything.