“The simple explanation is that physicists use math, but generally are not themselves professional mathematicians.”
Mathematicians live in a world of their own. During my graduate days, I became friends with one of my math professors. We each tried to enlighten the other. I think I was more successful than he was.
My brother was a math major who dropped out of college in his senior year because he said the math he was having to take had no practical application. He then joined the Navy for a six-year stint in radar. From there it was to Lockheed and GE Astrospace, where he supervised the launching of satellites. So the math may have been impractical, but what worked was lucrative.