Mine came over after (~1870) as well. In fact, the MAJORITY of whites today are descended from European immigrants who came over after the Civil war and thus never owned slaves. Of course even among those descended from bloodlines that came over on the Mayflower have ever owned slaves themselves, and they likely have bloodlines mixed with those who never owned slaves over the generations. I know, the obvious (progressive) solution is if you look white, you pay.
My ancestors came over between 1845 - 1869. None of them owned slaves and in fact, one of my great-grandfathers and his parents moved north from Missouri in 1860 because they hated slavery and their barns and animals were being attacked. They were German immigrants and none of the Immigrants from Germany supported slavery, regardless of whether they were Lutheran, Methodist or Catholic.