... and if they are the oldest civilization then shouldn't they be the most populous?
that doesn't necessarily follow.
At different points in time different areas have been the most populous.
Both the Ganga-Jamuna-Brahmaputra and the Yellow river-Yangtze regions are very fertile and support together about 1 billion people
But 4000 years ago they were thick with jungle and vegetation and not populated. At that time the temperate river systems such as in Sumeria, Egypt and the Indus valley were the most populous