God (or evolution or whomever or whatever ) created Man, with unalienable rights. Some of the Men in North America came together to make a written agreement to form a government to further some of their mutual interests. That written agreement is the Constitution and as ammended still applies. It grants the federal government certain power and authority, to accomplish certain common objectives. It places explicit constraints on that authority. It provides for specific structure and proceedings which that government must follow.
I don't need constitutional permission to choose to discriminate against fools, or whatever sort of person I dislike. So long as I don't infringe on your unalienable rights to progress in your life as best you are able, I am free to associate or not with you, and to do business or not with you, without any government authorization.
So, yes, literally, the Constitution does not allow discrimination in private affairs. But it doesn't prohibit it either, the wildly overstretched commerce clause perhaps notwithstanding.
Of course, my right to discriminate against you does not give me the right to affect harm on you, your property or your family. We have authorized our government to enforce laws that provide for the common safety and security of us all.
If you decide not to hire someone just because he is black or he is a homosexual then you are violating the most basic human rights that make the heart of the Constitution. And by doing so you are harming the person pursuit of happiness and his well being.