So let's dump language and say calculus. So some caveman from Quest for Fire had a need to work a second derivative to be sure the coal stayed lit, and hence the math center survived 100,000 years until Einstein came along.
Remember, homo sapiens sapiens coexisted with Neanderthals. But homo sapiens sapiens brains could work college math. How was the college math trait something which drove survival and propagation?
Well, math is mostly just logic.
Logic must have been useful early on in hunting and later, war. Incidentally, language is useful in exactly the same areas.
Put another way Calculus is merely an specific form of Abstraction, which is useful for say making plans, drawing maps, etc.