Japan actually has a policy like this, but they don’t make a big fuss about it.
People with valid visas have had them revoked and have been deported for stealing $50 bicycles.
If you are arrested for any crime, your visa status is checked. If you are an illegal, you will eventually be turned over the the Department of Immigration for deportation even if the investigation shows you did not commit the crime.
Quite frankly, I like it this way.
True story from the early 1990's. Since relations with Japan and Iran had been excellent under the Shah, there was a reciprocal visa free visitor agreement in place. As conditions grew worse in Iran, many of them used the loophole to escape the country. Some of them were good workers and got responsible jobs. Most of them ended up congregating in low rent housing and selling fake telephone cards to make a living.
Every time I passed through Ueno, Shinjuku or one of the other key transfer stations, they would approach offering to sell these cards. I got so tired of it, that one time I told them they needed to be more aggressive in their sales techniques, so I shouted "HENZO TELA-KADO HATSUBAI DESU YO!" They scattered like a bunch of illegals at a construction site used to scatter when somebody shouted "LA MIGRA!"
Not long after that, the immigration people started cracking down. It may have been a coincidence or it may have been that they were embarrassed by a legal GAIJIN calling attention to their benign negligence.