PSU has been a middle-rank team in the Big Ten, most years. They also haven't been producing pro players in the numbers that they had thru the mid-90's. Paterno was living off his reputation for much of that time.
Sandusky was a huge part of that reputation, that’s why when he was there, Penn State was referred to as “Linebacker U”.
If Sandusky was just some unknown assistant coach, I’m sure Paterno would have done the right thing, but this was a guy that was a huge part of Paterno’s success.