I think what may have affected the death toll even more than hitting other buildings would have been the inability to evacuate people from the towers themselves. On 9/11, the delay between the planes hitting and the buildings falling allowed many thousands of people to escape to safety.
Thankfully, the wall that separated the Hudson River from the subway did not fail. If that had happen the river would have flooded the subways tunnels.
Honestly, that slightly under 3,000 lost their lives in these attacks, ( Washington, PA, and NY) was somewhat miraculous.