Not to mention his role in the financial crisis which was devastating to the economy. Fannie Mae and Freddie Mac are not in a crisis...they are very stable financially.
You, and everyone else, should really
read this
To summarize: the Bush Administration came to Congress (specifically to Frank's Committee) and issued major warnings. Frank and his Democrat colleagues assumed that the Bush Administration was just playing racial politics (aside: about as clear a case of projection as I've ever seen) so decided to play politics right back.
Frank, in his most detailed explanation to date about his actions, said in an interview he missed the warning signs because he was wearing ideological blinders. He said he had worried that Republican lawmakers and the Bush administration were going after Fannie and Freddie for their own ideological reasons and would curtail the lenders mission of providing affordable housing.