'in particular her mentally disabled sister Rosemary'
I remember hearing/reading that daddy joe had the sister lobotomized.
She was. There are three versions as to why. One is that she was mentally retarded. Another is that she had violent outbursts and was unstable. A third is that she was sexually promiscuous and that Joe feared a scandal from her getting pregnant. She was apparently slow, but prior to her lobotomy, she could multiply and divide, and kept a diary, and attended parties. After the procedure, she was incontinent and almost an infant.
It's a tragic and scary story. Apparently, Joe was told that the procedure would calm her down and make her more easy going.