The population rose when people settled down to farm. It meant more disease and grinding labor, so farmers didn’t have life easier than hunter-gatherers. When there were fewer people around, there was less competition for berries and game.
Humans were bigger and stronger when they were limited to hunting and gathering. The shrinking man, that is evident in the archeological record as we moved away from hunting and gathering is evidence of that.
I’m not saying that the move to farming was a bad thing, but it did cause problems by doing so.