The building was actually mostly standing as recently as a couple hundred years ago. If I had to make a wild guess, it would be to blame the Turks.
That is to bad, having visited some of the worlds most amazing ruins (parthenon, and more) it is truely a shame not to preserve all ancient sites as much as possible.