Because they were his competition for the same crowd of radical followers
Like I always said, Nazis vs Commies was like Bloods vs Crips. No real differences between them, except for the uniforms.