This is why many small countries no longer accecpt bank deposits from Americans. Because while no American is subject to attacks from foreign tax agents, No non-American state is safe from the over reaching IRS.
You may be outside of the US, but you are still US property. Its a land of the free thingie.
That's a route around this problem.