People who profess any religion, or none, can be good conservatives in principle, although there are some interesting subtleties to be considered.
In one sense, atheism, in its unsentimental rejection of that which lacks empirical evidence, and its embrace of the reality of nature, is a good template for the rejection of much of fuzzy-headed bleeding-heart liberalism which is (it should be acknowledged) a secularization of a particular strain of 19th/20th century Christian charitable impulse.
In another sense, though, atheism, at least its more evangelical forms, is hard to square with good Burkeanism — a good Burkean is loathe to tear down institutions (such as churches) which have been bulwarks of stability and inculcation of socially-useful values and practices for many generations.
The only reason that a reasoning atheist would want to tear down a social institution would be if he felt the institution had grown so fundamentally flawed as to represent a threat to the greater social order, in that sense the atheist can be conservative in his belief that conceptual and institutional structure is necessary for society to grow and thrive, and at the same time be opposed to certain institutions, including certain forms of government, that are antithetical to that survival.
That said, it’s important to remember, and to interpret the founding documents in light of the fact that, although the Founders we not atheists, in the broad sociological spectrum of the day, they were somewhat radical, albeit gentrified, and would not have been considered fundamentalist or even mainstream Christians. The were mostly Deist, and its interesting to note how the language of many of the founding documents talk about God as an existing entity, but not an intervening one. They seem to have been careful not to talk about the providential and ongoing concern of god in people’s lives.
The founding documents tend to stress the fact that God made everything, and then left everything alone. Notice that man was created, past tense, and endowed with certain rights. This is the Deist view of God creating everything and then leaving each human to work with the gifts he is given. So they were not theological authoritarians or even Calvinistic for the most part, but more had the view of God as a creator and clockmaker, leaving man to find his way through the use of reason, a view shared by certain modes of modern atheist thought, although the larger group of atheists contains some unsavory characters. A reliance on reason makes modern atheist libertarians the intellectual inheritors of the tradition of the Deist American Founders.