Many huge houses are white elephants. People don’t think of the operating costs like heat, air conditioning, taxes, insurance, etc. At some point they need painting and re-roofing. It is one of the reasons the Biltmore estate home was donated.
Even at a much smaller level, say a 4,000 to 5,000 square foot house, these expenses can require substantial income. In 15-20 years there is going to be a need to repurpose a lot of McMansions.
You’re right; here in north Jersey we have the Palisades cliffs overlooking NYC, and wealthy people built mansions there. Over time the heirs gave them away; they were maintenance and tax nightmares. Eventually a Rockefeller (IIRC) bought up the land and donated it to a park system with the stipulation that any homes visible from NYC be torn down (couldn’t mess up the NYers’ view of the woods, I guess). I believe the only surviving mansion is in a ravine (invisible from NYC), and it serves as the park headquarters.
This battle has been renewed as companies develop land a short distance from the cliff and park; they want office buildings, while preservationists want one-story shacks built instead...