I worked with an old black man when I first got married at the ripe old age of 19. We both made minimum wage (about $2.35 hour at that time). He was in his sixties and had put two daughters through college. This was way back before all of the programs they have today. I still remember his name today. He was one of the nicest, most honest, and hard working guys in that entire farm equipment dealership. I and everyone else always used mister, when addressing him.
I suspected you were from the South, probably rural South. I was raised to refer to my elders as Mister or Miss and their first name, all ladies 16 and up are Maam. Race didn’t matter.