That is true, but unfortunately, one of them isn't us. I don't see why it would be so bad to do a little planetary engineering. We can get rid of some of the potentially killer asteroids by ramming them into Mars. Ram them head on into Phobos or Diemos, and the whole ball of wax will go spiraling down onto the planet. There's nothing there to break. No fragile ecosystem. No people. No cities.
It'll give us some practice moving asteroids around and at the same time make Mars a better place to live.