I believe it specifically that there will be no official religion sanctioned by the government.
The context of the establishment clause, as all good FReepers well know, is the Church of England.
When you want to know what not establishing a religion means, look to the C of E, and say “not that”. A coach saying a prayer before a game is not establishing the analog of the Church of England.