People will just move their wealth abroad, only hurting the country. If you then go after those tax shelters, they'll move themselves abroad as well, and remain out of reach.
let them go. you can't use that argument to justify massive taxation of those who work, and low taxation of those who relax all day.