It was certainly clumsy as public statements go. I detest political correctness as well but I think there were better ways to express it.
I honestly think he misspoke and even if he didnt I hold it as no worse than the numerous racist statements said, without consequences, by members of the congressional black caucus.