Yes, unless you turn so that people within earshot can see it, you sound crazy.
I remember when the Bluetooth cel phones first came out and there was an army of crazy people walking around here seemingly raving at themselves. I really got a kick out of the ones who couldn’t talk without waving their arms around. It was like a prolonged Monty Python skit. Now I see people talking to themselves and I don’t think anything about it. I thought about that several months ago when I saw a show on TV about this guy who was in a coma for 19 years or something, and woke up. He was in for a number of big surprises.