Slavery was a system of labor which brought profit to the slaveowners, but it was also a system of racial control—”white supremacy”—which most whites believed in. Lincoln abhorred slavery, but at least in his public comments supported the idea of the white people being on top in society. Even Northern whites who rarely saw a black person worried about what would happen if the slaves were all freed and came North to compete for jobs—and most Northern states did not let free black men vote.
His thinking on the matter of racial equality evolved over time. Toward the end of his life he became much less of the white supremacist. IMHO, it was likely because the valor of black soldiers during the war impressed him.
Even Northern whites who rarely saw a black person worried about what would happen if the slaves were all freed and came North to compete for jobs—and most Northern states did not let free black men vote.
That changed with the ratification of the 15th Amendement.
How do you explain the thousands of black slave owners? And the black slave breeders who sold their own children into slavery?
I guess it wasn’t solely a white supremacy thing.