The simple answer is that there are at least two sides. These two sides disagree on how society should be ruled. Neither side is willing to accept the rule of the other.
It is “force”, one idealogy against another.
I'm going to quibble with this:
Our side says: Leave me alone. If you leave me alone, I will leave you alone.
Their side says: No. I will rule over you. I must control all aspects of your life.
There are two sides, but only one is demanding to rule over the other.