I hope that is not what D'Souza said.
Over 2 million men served in the Union Army during the Civil War and well over 300,000 died during the course of that war. Of those deaths, approx 2/3 of them were from various "camp diseases" while the other 1/3 were killed in action on the battlefield or died as a result of wounds in battle. Another 200,000+ were wounded in action but survived.
Taken together, there were over 500,000 Union casualties during the war, or approx. 25% of the 2+ million man army.
Well they must have died only because they hated blacks and wanted to have them all sent back to Africa. :>}