This was a common view in that many early Americans considered themselves Virginians (for example) first and Americans second.
That wouldn't have made them anything OTHER than natural born citizens of the USA.
Nothing about being a natural born citizen entails perpetual allegiance. Robert E. Lee for example was a Virginian in his primary allegiance - and an American secondly - and not perpetually aligned.
Well, no Madison specifically said the King could make them into denizens, but not natural-born subjects, and that the legislature would need to pass a naturalization act to make them British subjects.