Apparently what they had on Roberts involved his adopted children and how he got them into the US. At least that is what I kinda remember.
There were rumors at the time (and re-surfaced when 0kakacare got into the SCOTUS) that he was a homosexual either in the past or still. A photo of him and some other guys at Fire Island was posted, he also did pro bono work for Lambda, a notorious homosexual advocacy group. He didn’t marry until age 40 and not sure if there were other girl friends before that.
Odd....