You are most probably right. The opposition doesn't want to be vilified and alot of Christians hide behind "the Bible tells us not to judge others".
You're right. We should answer by saying, "I'm not judging people, I'm judging behavior, and I'm using the Bible."