Lot of fans forgive Frank's fraternization with the mob saying he had to pal around with them b/c they owned all the clubs where he sang to make a living.
It's pretty hard to work in those clubs without brushing shoulders with them, but of course saying hi to them in a nightclub is obviously a different degree of fraternization than running a casino for them, etc.