Sounds as if you remember the real Abercrombie and Fitch, which was NYC's answer to LL Bean, only better. They even had their own shotgun brand. It was one's last stop before leaving on a real big game safari.
Started coming unglued when some desperate clerk walked in one day with his own load of #4 buck and offed himself in the store. Bingo. No more new and used gun department.
Canoe? No problem. X-country skis, snowshoes,hiking boots, tents, and really good, stylish country clothes for the real people who populated the US at the time, not sleazy crap for today's metrosexual ani.
There was also the Folbot store further downtown. Got one for my 9th birthday! Worked great for a long while.
Oh yeah, and big game guns. You are correct. Some mighty fine weapons. Now they’ve devolved into a hot steamin’ pile. No silk panties for men back then. LOL! Never went there but I have read about some of the fine weapons.