The same thing happened in the late 1300 to early 1400s period ~ Black Plague put a stop to that growth stuff.
Wars have become less deadly BTW. Back in the day they were far more deadly.
Then, there's the early 1500 to mid 1600 period in the Americas where almost the entire native population of tens of millions of human beings were destroyed by hanta virus.
The argument about contractions fails to take into account that, when you assume a large age, like 100,000 years, then the few years of negative growth become even more insignificant when looking at the aggregate growth rate. If we were only 5,000 years old, then the periods you cite are significant, but at 100,000 years old, a century or two of negative growth is negligible.