Only Congress can decide on the qualifications of its members. There is no constitutional provision for a voter recall of federal officials.
"Show me just what Mohammed brought that was new, and there you will find things only evil and inhuman, such as his command to spread by the sword the faith he preached." - Manuel II Palelologus
Except that when there is no provision in the Constitution, such as a recall, the 10th Amendment clearly delegates those powers to the State.