No doubt. But this, honestly, is a feature. I have a Nikon SLR. To get it to record GPS, I have to buy an attachment. There are many other configurations and software that tag and read photos using GPS.
The purpose for photography is to be able to return to the spot as well as describe the photo by exact location.
Since phones have both GPS and a camera, it would seem natural to combine them. I'm only surprised they don't turn it off so they can charge extra to turn it on.
I wouldn't have known about that yet had I not read an article about it. That photo I experimented with happened to be of my grandson. Imagine what a perv or kidnapper could do with such information.
People plaster photos of their kids and family all over the web, especially on networking and sharingsites like Facebook. I warned my family that what you choose to reveal about yourself is one thing, but your network of "friends" can reveal more info about you, and it opens you up to all sorts of criminals and stalkers, etc., knowing when you'll be on vacation, going out for the night, etc.
That was cool how you got your phone back. The police don't have the time to handle these myriad of complaints over what would be considered petty theft, and likely wouldn't have done much about it anyway in your case. So justice was served. But I'll bet you never think of that particular phone in quite the same way again. You taught them a good lesson.
Somebody stole my gdauther's cell phone, and it always made me sad and MAD for her because she lives on the edge financially but is paying her own way. She never got it back. So I don't know if I'd go that far or not, I could be pushed to, but that was clever how you got your phone back.