Definitely John Adams, John Quincy Adams, Millard Fillmore, Franklin Pierce, and James Buchanan were never slaveholders. Martin Van Buren's father was a slaveowner at one point but I don't know if Martin himself was ever the legal owner of another human being.
Likewise with William Henry Harrison--his father, Benjamin Harrison, was a Virginia plantation owner, but he joined the army at the age of 18 after his father's death and spent the rest of his life in the Northwest Territory, Indiana Territory or the State of Ohio, except for a year as US Minister to Colombia or living in Washington as a Congressman or Senator (and for one month as President). The Northwest Territory was closed to slavery before he went there. He may have been a legal heir to some of his father's slaves (he was the youngest of 7 children) but that may have depended on his father's will (if there was a will).
The records show 5 ~ but there are 2 others out of that first 15 who didn't own any slaves while they were in office.
George Washington got Virginia's statutes revised so an owner ~ himself for example ~ could free his slaves in his will, which he did.