Same here. Don’t think it was because of their skin color per se, it was that the skin color was not the same as ours—the owners.
Since darker colors absorb light and lighter colors reflect light, I assumed our dog had a difficult time seeing features of some people (and other dogs) and it caused him some anxiety. Our other dogs never seem to mind.