They were pushed west and ended up in East Tennessee and Southwest Virginia where they are presently known as Melungeons.
Origin of name obscure. Portaguese for "shipmate," turkish for "cursed soul." Sir Francis Drake liberated 500 levantine galley slaves working under the Spanish lash along the coast of South America, and brought 100 of them to England, along with the first batch of Roanoke Island discouraged colonists.
There were Portaguese settlements in the Carolinas in the mid 1500s, with trading routes that extended far inland.
When the Scotch-Irish folks migrated down "the big valley" from the North, in the highlands of Tennessee they encountered darker-skinned, Catholic folks who'd gotten there first. SO they decided to call the earlier arrivals "Free Persons of Color," with reduced standing under the law.