You are assuming that voters have final say in who their elected officials are. They do not.
The framers enacted a system where originally the only federal official you directly elected was a Representative. And if you need to fire them, you get to do it in two years.
Senators used to be voted on by state legislators, not voters.
And with Presidents we have the Electoral College which has been bastardized.
The Electoral College could have demanded proof of eligibility. Same with Congress when certifying the vote. Congress can still impeach him.
You are left without a direct recourse, but not without a recourse.
Again, think about the path you are advocating.
Want to stop the Iraq War, Code Pink? Sue George W. Bush in federal court.
Want to stop tax cuts? Sue Ronald Reagan in federal court.
It would be endless.
Sorry, this is a straw man argument because if Mr. Obama is not a natural born citizen, he by existing law cannot be president. It really is not about law suits beyond the people attempting to get him to provide proof of his legitimacy. If he can do this no further action is required. If he cant, we have a real problem.