"Oh, playing the race card? Weakest argument in the book. Now, as I was saying about the deficit ..."
You could also ask the accuser to cite where the constitution excludes the rights of racists.