You make an excellent point: a cohesive, highly motivated minority can defeat a majority which is mostly unmotivated and divided into factions. For generations now, any talk of "white solidarity" has been condemned as evil and racist.
Also, any conflict that starts out black-vs-white is quite likely to devolve into Right-vs-Left.
According to uselectionatlas.org "Obama" received (rounding) 69,500,000 votes nationally in 2008.
If only 1/2 of 1% of that number were to feel compelled to engage in mayhem, that would be 347,500 people all over the country wreaking havoc in some form or another.