Yes, it is a fact that he was not charged, and it is a fact that police found no evidence to contradict Zimmerman's account. The speculation to which I am referring is the many stories about exactly how they came to be face to face, and who initiated the fight.
For example, a lot of commentary and news articles repeat that Zimmerman "chased" Martin, which I do not believe fits the known facts as I have summarized them above.
To the contrary, though, I think one could just as easily conclude from the map and facts you have provided that Zimmerman not only followed Martin but also “chased” him (”Martin decides to run”) until losing sight of him.
Martin, on the sidewalk, rounded the apartments and started heading south towards his home. Zimmerman spots him again. It appears to me that Martin then stopped and confronted Zimmerman. And that’s when the SHTF.
But whether Martin walked up to Zimmerman, or the other way around—or if it was a `mutual combat’—I suppose there’s the nub of the matter. The fact remains, however, that Zimmerman had the gun.