Do you have a constitutional reference for that? I can't find one.
but the Constitution gives each house the power to determine the qualifications of its members.
This is the relevant line here. However you may interpret it it has been interpreted to be that the Senate can determine the fitness of a Senator to serve and can refuse to seat him. I am sure it would take at least a majority vote and is unlikely but it could be done. USSC would probably have no jurisdiction here.