The stone in question is probably more like 4000 - 10,000 years old and Julius Caesar probably weighed about the same then as now but, yeah, that's the basic idea. Give you another example: in today's world, the biggest birds which can take off and land are albatrosses at around 30 lbs. and they can barely take off and land. Their takeoffs in fact are so problematical that sailors generally refer to them as "gooney birds". In all larger birds, the wings have become vestigial, and the birds no longer think of flying. In some recent past age however, the Argentinian teratorn flew; his wings were for sure not vestigial and nothing with wings like that could live other than by flying:
The teratorn, as you can see, was about 200 lbs, with a 25' wingspan; nothing like that could fly in our present world. The 8 - 25 million year thing which the article mentions is based mainly on uniformitarian assumptions.
Since you posted the reference to Baalbek, I assumed you were somewhat familiar with its history. To quote from the Catholic Encyclopedia:
Among the monuments of Baalbek were three temples: the Great Temple of Jupiter, the Temple of the Sun, and the Circular Temple of Venus; all of them date from the second century A. D.
I suppose it is faintly possible that the giant stones you refer to were already standing and the rest of the temple was built around them, but that seems fairly unlikely.
BTW, do you know how these stones compare in weight to the obelisks which were routinely moved and raised by the Eguptians?