*** 500 years later a late comer group calling themselves Muslims build a mosque and deny that the Romans were ever there. ***
A little more complicated. The second Temple was destroyed by the Romans, who later built a temple to Jupiter on the site. This was destroyed to make room for a THIRD TEMPLE built by Julian the Apostate. It was destroyed while under construction.
Then came the moslems.
There was a Church to Mary there for a while also. I did leave out the Jews. They were in the area for centuries also.
If it was prophecies there will be third temple at former site by the wailing wall that include where muzzie temple now, G-d will always make a way that will come to pass, just like how Israelites conquered their promise land from the Canaanite squatters.