Aren’t the muzzies descendants of the Philistines? It could be that they started shrinking after Goliath was beat by David.
I kept scrolling down the comments to see if someone would say what you did - I was thinking the same thing. Goliaths, descendants of the Nephilim.