I agree with you.
In 2007, Vanity Fair magazine published "Mad About The Boys" detailing extensive allegations that Pearlman molested some of his young boy band members. Pearlman publicly addressed the allegations in a 2014 prison interview with The Hollywood Reporter magazine, claiming they were unsubstantiated, and that he'd never been accused of such by anyone he'd helped succeed. Vanity Fair, Pearlman claimed, had only interviewed "people that had a grudge."
The Hollywood Reporter noted Pearlman's "well-documented track record of robbing his artists blind" and rumours of his predilection for "buff, blond boys in his entourage", while a ten-year relationship with a former nurse, Tammie Hilton, remained unconsummated. "He was very religious," Hilton said.
However, in a 2009 interview with Howard Stern, the former lead singer of band LFO, Rich Cronin, said that he'd only received a fraction of the money owed to him from record sales, and claimed that Pearlman had "wanted to bang everyone," and had attempted to seduce him multiple times.