I might be wrong but in Casino Royale when they put him on the chair and tortured him with their novel technique(Belgium’s)?.
I always got the idea that he became sterile because of the torture and was depressed,so he played around and when he did fall in love she was taken from him.
His first love committed suicide going into the water and never coming back alive.
All that is left is a very angry depressed Bond who buries it all by fighting for his country without care whether he lives or dies.
Le Chriffre threaten to cut his junk off after torturing his testicles with a carpet cleaner. No mention was ever made of sterility. However, you forgot in YOLT Bond gets Kissy Susuki pregnant before heading off to Vladivostok.
At the end of Diamonds are Forever he wanted to marry Tiffany Case but by the time FRWL roles around, she leaves him for an American Marine. Vesper Lynd killed herself with pills and he was angry, he was angry at Vesper “The Bitch is Dead” and he was angry at Smersh.