Yes,
His father side is German (immigrated in late 1800), his mother was poor Scottish cleaning lady, immigrated in early 1900.
But heck, there were native slaves in Europe too.
That ended around 15th century, but the serfs were not too much better off than slaves. French revolution and the revolution of 1848 ended most servitudes in Europe, except for Russia (1861).
Turkish empire, which controlled southeast Europe, abolished slavery like 1922.
If you include all of history 100% of the world population is descendant of slave owners. Yes all of us. Obviously they were only talking about USA and colonial American slave owners.