Could be, but still - humans are not ants, FRiend.
You build buildings you are going to be cutting down trees. Cutting down trees was very dangerous. So much so that an axe head flying off and killing one of your fellow tree cutters was the example given in the Books of the Law for an accidental homicide.
And that was just the chopping. You add in hewing, transportation, lifting and people are going to get hurt. And that is building with wood. Building with stone and brick was even more hazardous.
In contrast with the HG life style you are felling saplings at most. Less chopping, less chance for injury.