This is a silly idea Physical taking, merely by itself, is no basis for ownership.
It is not physical taking, merely by itself; that would be theft or robbery. In this case, the act of taking is also associated with the act of defending oneself against a potentially deadly attack, i.e. stealing the gun from a would-be murderer, while in the middle of the act of attempted murder, of the person in question who is taking the gun. Your answer misses a very important part of the question.
I have never considered the question before, but my answer is yes, one should be allowed to keep the gun.
A separate question is whether the police should have any right to attempt to connect that gun to past crimes (as by taking ballistic evidence in the form of test-fires) or anything else.
Yet another question is what should the citizen’s rights be, if the gun is prohibited, as by having a defaced serial number. If that is the case, should the citizen have a right to win a suit against the criminal for the value of a comparable, legal gun plus lawsuit costs, if he wasn’t allowed to keep the actual one as a result of its illegality? (Yes.)
“Physical taking, merely by itself, is no basis for ownership.”
Are you able to read?