Weren’t there some Giants among the Canaanites?
I’m surprised this hasn’t been suggested before. Gezer was an important and well-known city long before the time of Exodus and Judges.
Also a convenient location for Israelites, being halfway between the port of Jaffa and Jerusalem.