Something similar happened to me during this past Christmas season. I was making my way to the car with my purchases. A young man approached me while I was loading the car with some sad tale. I kept the cart between us and opened my coat to get my wallet. In the process of showing him I was not carrying cash, my .45 was not-so-subtly displayed. He went elsewhere.
If I'm reading your post correctly, you did something beyond foolish.
For a time I was working a part time telephone job in a bad location, the business has moved since then. One morning I pulled up at the old location and as I stepped out of my F-150 I saw a typical panhandler type who was walking across the parking lot turn toward me. He walked straight toward me at a fairly rapid pace, he looked to be in his thirties, about six two and healthy, not the sort you normally want to have approaching you. I am an old man but I am big, six four and around 260 pounds and I am not fat. Ordinarily I would have felt very uncomfortable because I did not have my weapon in my pocket as I normally would but that day I was in a rather strange mood for some reason. I stopped, looked straight at him and he started some BS approach, asking me how I was doing as if he knew me. By then he was about ten feet from me. I asked him in a very grouchy tone what he wanted and he stopped in his tracks and started speaking very politely as he turned and walked on the way he had been going before I pulled up. I don’t think he liked the look in my eyes. I was already thinking that if necessary I was going to see if I could drive the heel of my hand straight into his nose and stomp on the arch of his foot at the same time. I figured that if I succeeded the next thing I would need is a good lawyer and if I didn’t who knows.
I suspect that his game was to approach in a such a way that most people would be intimidated by his size, he could see that I was bigger than he was but also twice his age or more so he figured I would rather give him money than take a chance but he changed his mind. Panhandlers used to be unknown in this area but they are becoming more and more common.