“we could all be dirt poor peasants before we get a chance to escape”
That is a common misconception. Even if you could invent a Star Trek transporter, there are too many people to transport. The remaining population on the Earth produces more children faster than the transporter can transport them. The escapees can only be the descendants of the small number of people who colonize the new extraterrestrial habitats. The population of humans left behind will become extinct, sooner or later, leaving the emigrants as the survivor population/s.
Of course only a small number would ever be able to actually escape. If I developed the technology, I would tell nobody and I would hand-pick who comes with me.
But perhaps it's much more difficult to adapt to living on even the most Earthlike words we find - so much so that on each new planet we have to take some of the native lifeforms and start incorporating their genetics into the humans who will become the first generations that can survive on the surface. The rest of us have to terraform underground bases that maintain air that is more Earthlike, grow Earth foods, and shield ourselves from the constant alien microbial onslaught, all the while perfecting and adapting our species to life on that particular planet. If there's a moon, perhaps set up underground shop there, first.
Because of the constant threat of meteor strikes, gamma ray bursts, or just plain nasty animals with big shreddy teeth, advanced civilizations may prefer building underground and terraforming "up".