If consent were the only basis of government, I would have the right to discard any law I didn't feel like obeying.
True, the Declaration of Independence states That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed, but we are still faced with a "nebulous third party:" the God which endowed men with those rights.
Also, that "consent of the governed" clause is tricky: if we take those two words literally, no government action could ever be taken without unanimous consent of the people.
However, seeing as how rights are "secured" against people who would otherwise violate rights, it's clear that government is a necessary "nebulous third party," which is empowered to operate without unanimous consent.
"Consent of the governed" is vital, but it's not absolute.