So is he saying that all real conservatives must quit the congress like he did?
No he's definitely not saying that. He encouraged Niki Haley to appoint TEA Party backed Tim Scott to replace himself and after a short time of pondering, she did.
Scott, a young black Representative and now Senator, is very Conservative.