Start with that.
Dinesh D'Souza debunked that already. Julia Dent, President Grant's wife was from St. Louis, Missouri where her parents owned around thirty slaves and of course they were Democrats. General Grant got a slave from his father-in-law in 1858. He freed the man in 1959 and when he owned his slave, he was a Democrat! In 1856, he even voted for the Democrat Candidate James Buchanan.
It is cutting it pretty close to try to leave the impression Grant was not a Republican and that he never owned slaves.
That kind of smart-guy thinking led to D’Souza’s felony conviction.