Like the antiquities of the Americas and Rhodesia, the local people knew nothing of these structures and attributed them to long-vanished "ancient ones" or even "gods", "celestials," "star people," etc.
This even if conceivably their own remote ancestors DID build them. The peoples had already long since lost their own history, if they ever had it, before the West ever arrived. The West if anything, is the only reason they DO now have any knowledge of it.
In all honesty, I doubt those structures are more than 500-700 years old... maybe a 1000 tops, but I could be wrong... :0)