Exactly.
Actually, I would get the government out of a lot more than just housing.
With practically everything that the Federal Government touches--ranging from college tuition to housing to transportation--the Federal Government is notorious for destroying the quality and quantity of goods and services delivered and dramatically increasing costs and thus prices, relative to the "normal" pull and push of market forces like supply and demand. Simply put, to paraphrase Reagan, the nine worst words in the English language are almost always "I'm from the Government, and I'm here to help."
How far do you want to carry this ~ I'm game!