The key part is that you know they are armed.
An armed society may be a polite society, but when politeness doesn’t happen, it gets resolved pretty quickly, and pretty finally.
The intent of the quote is that armed people know they are armed and thus avoid starting arguments and such.
An armed man walking around with a chip on his shoulder is the bad guy, and is screwing all of our rights.
Numerous episodes of Gunsmoke, The Rifleman, Bonanza, and Wagon Train explain this concept in Shakespearian detail.