The Framers (and their successors in American government) allowed certain types of non-religious-non-political speech to be banned. When you have an explanation for that which fits your assertions, get back to me.
Please answer this: Do you doubt that the term "licentiousness" meant some degree of vice/immorality in 1789, regardless of whether it included "obscenity"?
Madison flat out said the First Amendment protected licentiousness, which means he believed the First applied to more than just political and religious speech. I don't know how you can argue otherwise.
The Framers (and their successors in American government) allowed certain types of non-religious-non-political speech to be banned.
What are some examples of Framers "allowing" Congress to ban such? Did Madison approve?