When Bill Cosby said something similiar, the Blacks went off on him...hopefully they will for Obama too.
"The Blacks" are not monolithic. There was certainly a considerable backlash, but he also got support from a number of black leaders, Jesse Jackson among them. Cosby wasn't run over by a train of public opinion -- he sparked a debate.