One possibility might be a large earthquake that liquefied the subsurface causing violent venting among other theories:
In the New Madrid region, the earthquakes dramatically affected the landscape. They caused bank failures along the Mississippi River, landslides along Chickasaw Bluffs in Kentucky and Tennessee, and uplift and subsidence of large tracts of land in the Mississippi River floodplain. One such uplift related to faulting near New Madrid, Missouri, temporarily forced the Mississippi River to flow backwards. In addition, the earthquakes liquefied subsurface sediment over a large area and at great distances resulting in ground fissuring and violent venting of water and sediment. One account of this phenomena stated that the Pemiscot Bayou "blew up for a distance of nearly fifty miles."
If you're talking about the bays themselves there's actually a good deal of overlapping in a lot of places. Even bays within bays in some instances; overlapping rims, etc. If you have or can get Google Earth you can spend hours scouting around the planet looking for anomalies and unusual formations. Some of it VERY cool.
BTW, the liquefaction possibility is an idea I floated several years ago in earlier discussions but I haven't seen anything to about it one way or the other since. Oddly enough or not, our east coast is apparently not the only place in the country, or the world for that matter with bay formations or oriented lakes, etc.