When I go hiking, I always keep my loaded .44 Magnum in my backpack. That would stop a charging grizzly bear.
You might consider a good chest holster.
Getting it out of a backpack might take too long.