Only citizens at the time the Constitution was passed were eligible ... every one after that point had to be a “natural born citizen”
There is a huge fly in your argument:
You don’t know the true identity of his father so it renders your point moot.