I'm going to quibble with this:
Our side says: Leave me alone. If you leave me alone, I will leave you alone.
Their side says: No. I will rule over you. I must control all aspects of your life.
There are two sides, but only one is demanding to rule over the other.
The counter to your quibble:
From the view of the left, the Right is already in power and already rules over the left. So from their point of view, the Right is controlling all aspects of their life and that is what they want to overturn.