All of our rights are bracketed by the existence of our fellow-citizens rights. (Free speech and yelling fire in a crowded theater, etc etc). This should be self-evident. To argue against that is to argue you have superior rights to everyone else.
Morally, that's true. But it's the job of the law to determine when our rights violate the rights of others. To argue otherwise is to argue that we don't need laws at all, other than "just make sure you don't violate anyone else's rights." Sounds beautiful, yes, but I think society might run into a bit of trouble unless its laws got a little more specific than that.