Source: Letter of Thomas Jefferson to Dr. Thomas Cooper, February 10, 1814.
Blackstone, who compiled the English Common Law would differ. Adultery was a crime under common law and for many years a crime in many states in the US. Check the 1892 US Supreme Court decision that refers to the US a “Christian nation”.