As popular and pervasive as the flag is, if it was really a symbol of racism, then racism would be as popular and pervasive as the flag is.
A little logic can do wonders.
Most people think generic South, Rebels, maybe Confederates.