That poor taxi driver sounded terrified! If I were a taxi driver, I think I would try to be selective and keep driving past some possible clients.
And you can’t do that, legally. Pass someone up because you don’t like his looks, and he can report you to the taxi commission, and you lose your hack license.
What you say had better be a whole lot more specific than “He was a young mean looking guy with a bad vibe.” Something like, “He was baring his teeth, and there were wet red stains on his shirt, and it looked like he was carrying a machete or something.”