Notice that Danny Glover treats all whites the same. The ones in the north who ended slavery peacefully and the ones in the south who didn’t. The ones from the north who fought a civil war to end slavery and the ones in the south who fought to defend their region from what they viewed as an invasion. The ones from the south who, despite the urge to protect their home from an invasion, joined with the Yankees in order to end slavery. The whites who arrived here after the Civil War was over. What do all those people have in common: their skin color.
The north fought the war to prevent secession of all but four slave states. The Emancipation Proclamation was a military document intended to foment slave revolt in the Confederacy only, it did not free slaves in the four Union slave states, who in fact were not freed at the end of the war. The slaves in the four Union slave states had to wait for passage of the 13th Amendment to the Constitution of the United States in order to be freed. The four Union slave states were Delaware, Kentucky, Maryland and Missouri. There were over 450,000 slaves in the 1860 census, in Union terrirory. Washington, DC itself didn't even make slavery illegal until after the Civil War had been going on for a year.
It's important to get your history right. Popular belief is often incorrect.