It's not quibbling. It's the whole point. The second amendment either protects arms for all the citizens or it protects arms for the militia. Which is it?
You cited Federalist 46 and implied the whole citizenry was protected. If you're now saying that you didn't mean everyone, then fine. We're in agreement.