Bad conditions would imply that those left behind were even more negatively selected. In fact, the far higher genetic diversity within Africa suggests that the emigrants were the marginal non-dominants being outcompeted for resources and access to mates.
Any polygamous species always has a “halo” of beta- and gamma-males who can’t challenge the alphas who exist on the margins of the main herd’s habitat.
It stands to reason that at some point a number of these betas, with what few women they had, opted out of competition with the alphas and decided just to strike out into entirely unknown territory.
So, when you get down to it, everyone complaining about exploitation and colonialism is complaining about payback.
If you look far enough back, everyone was a master, and everyone a slave.
The genetics in this article might also be explained by an interrelated group ousted and attracting the beta and gamma males along with it. The females were more related, and the ousted group took the help. Or when they had become established in a new territory, a new generation of ousted males were accepted for genetic diversity.