Newt has some disturbing skeletons in his closet. And he is one of the few Speakers to be removed on ethics violations.
He was not removed for ethics violations. He resigned after the 1998 elections, because some house members wanted to replace him as speaker since the Clinton impeachment turned badly on them.
And his one "ethics violation" was a joke. He got caught up in a snare because he taught a college class about conservative policies that got characterized as a political training session, and the democrats trumped up a false tax issue because it was paid for by a tax-exempt group and they argued it was partisan.
I'll let Wikipedia explain it:
During his term as Speaker, eighty-four ethics charges were filed against him; eighty-three of them were dropped.[61] The remaining charge concerned a 20-hour college course called "Renewing American Civilization" that Gingrich had taught through a tax-deductible foundation, Kennesaw State College Foundation. Allegations of tax improprieties led to two counts "of failure to seek legal advice" and one count of "providing the committee with information which he knew or should have known was inaccurate" concerning the use of a tax exempt college course for political purposes. To avoid a full hearing, Gingrich and the House Ethics Subcommittee negotiated a sanctions agreement. Democrats accused Gingrich of violating the agreement, but it was forwarded to the House for approval.[62][63] On January 21, 1997, the House voted 395 to 28 to reprimand Gingrich, including a $300,000 "cost assessment" to recoup money spent on the investigation.[64][65]The full committee panel did not agree whether tax law had been violated.[66] In 1999, the IRS cleared the organizations connected with the courses.[67]
He resigned, IIRC.