I suspect there were White House Bible studies in the early days of this republic, and nobody was upset by them.
“From Jefferson through Abraham Lincoln, many presidents attended church at the Capitol; and it was common practice for Members of Congress to attend those services. For example, in his diary entry of January 9, 1803, Congressman Cutler noted: “Attended in the morning at the Capitol. . . . Very full assembly. Many of the Members present.” 16 The church was often full “so crowded, in fact, one attendee reported that since “the floor of the House offered insufficient space, the platform behind the Speaker’s chair, and every spot where a chair could be wedged in” was filled. 17 U. S. Representative John Quincy Adams (although noting that occasionally the “House was full, but not crowded” 18) also commented numerous times on the overly-crowded conditions at the Capitol church. In his diary entry for “February 28, 1841, he noted: “I rode with my wife, Elizabeth C. Adams, and Mary, to the Capitol, where the Hall of the House of Representatives was so excessively crowded that it was with extreme difficulty that we were enabled to obtain seats.” 19 Why did so many Members attend Divine service in the Hall of the House? Adams explained why he attended: “I consider it as one of my public duties- as a representative of the people- to give my attendance every Sunday morning when Divine service is performed in the Hall.” 20
There were plenty of them not that any one who has been to school in the last 50 years would know.
The first printing of the Bible by Congress was printed and recommended for use in the schools to teach reading.