"It is an established maxim that birth is a criterion of allegiance. Birth however derives its force sometimes from place and sometimes from parentage, but in general place is the most certain criterion; it is what applies in the United States; it will therefore be unnecessary to investigate any other. Mr. Smith founds his claim upon his birthright; his ancestors were among the first settlers of that colony."
In general place is the most certain criterion especially when that place is South Carolina where one's ancestors were among the first settlers of that colony.
No wonder Madison was so certain of his allegiance -- Smith was a natural born citizen of that colony.