Libertarians usually just want government to keep the peace and fix the roads and stay out of everything else.
A common misconception. Because they created a Federal government with delegated powers doesn't mean that was their view of all government. They had established churches. The laws in the states in the early Republic would be anathema to libertarians. The closest thing to libertarianism at the time was Jacobinism. It was feared and hated. Paine endorsed it, almost paid for it with his head and was shunned by nearly everyone as a result of his ideology.
The only reason that libertarians do not call themselves "liberal" is that Progressives somehow appropriated that word in the early 20th century. Until then "liberal" referred to the philosophy of John Locke, which was indeed the philosophy of the Founders. Certain key concepts reasoned by Locke form the basis of the American idea of government and the rationale for its Declaration of Independence, and are incorporated directly in its founding documents. The Founders were liberals in this sense and their opponents, the Tories, were the conservatives.
Unfortunately, the language of politics has become so corrupted that it almost seems as if it were "designed to diminish the range of thought", which was the objective of Newspeak in Orwell's 1984. The words "conservative" and "liberal" have no fixed definition and mean precisely what the speaker (or hearer) intend them to mean, nothing more nor less. Unfortunately, the word "libertarian" has recently been similarly corrupted with knuckleheads as diverse as Noam Chomsky, Bill Maher, and Glenn Beck referring to themselves as liberatarian, or leaning libertarian. It may be somewhat correct for Beck since he's at least superficially partial to the Founders, but Chomsky and Maher's views align with the Founders and "real" libertarians in very few regards and only by coincidence.
"The end of Law is not to abolish or restrain, but to preserve and enlarge Freedom." -Locke