Once they fell I always thought the same thing.
And the reason the second tower that was hit was the first to fall.
The second plane hit lower on that tower.
The weight of the upper floors was to great after the fire weakened the area that was hit.
That, and it hit a corner. More structural weakening. Closeups of the collapse show the top of the building slightly tilting toward that corner.