That is the picture that the police detective in the book studied. Look at it. Do you see a hunch-back? The face looks to be that of a thoughtful and somewhat pensive man, not a raving villain. Even allowing for the prettification court portraitists did, that is not the portrait of a cruel man.
By the way, the Tudors had custody of the two little princes.