The word public has a number of senses. First is public in the sense of "open to just about everyone." Second is public in the sense of state-owned. Third is public in the sense of common property. It seems to me you're conflating these meanings. Just because a property is privately owned does not exclude it from being public in the first sense.
Also the notion of law as a creature of tradition rather than a product of legislation seems foreign to you. This is unsurprising given the gutting of common law in recent decades. It is difficult for me to see how criminal justice requires property that is public in the second sense (state-owned) in order to function. Do we really require politicians in order to have law?
And what exactly do you mean by a double burden? Are you saying that you think a person could not purchase police services without the existence of state-owned property?! This seems so odd as to verge on non-sequitur. Perhaps I'm misunderstanding your point.
It is naive to think that common law implies a complete absence of state authorities. The fact that customary laws are not made by legislatures does not mean that there are not judges or magistrates or constables acting on governmental authority to execute those traditional laws.
By what right would you deprive someone of their liberty? You may say this is justified if one violates the rights of another, but on what authority are you acting? If someone refuses to accept the principles of your society, what authority do you have over them?
The question of the double burden: in a society with out public legal institutions, if I am completely plundered of all my possessions, I would not be able to get justice unless I can pay for it. That is quite an imposition.
It looks like having public entities to administer justice is not too high price to pay -- a form of insurance -- to avoid that double burden. Does the state -- and thus the taxpayer -- then bear the burden? Yes, but if the system works it will be cheaper in the end than relying on private enforcement.
Do I conflate the various senses of the word "public"? I don't think so, but as gated communities become more common it's easy to imagine some of these senses being conflated. While there are distinctions to be drawn, it is possible that some senses of the "public" sphere may be lost if the state were ever to disappear.