You are quite correct. However, I fail to see why the point is relevant.
There is much greater variability in strength and athletic ability, for example among women and among men than there is between the two groups.
Does not change the fact that men, as a group, are able to beat the crap out of women. That there are extraordinarily exceptional women who can beat up most men, and men who most women could beat up, also does not change the basic fact.
ka-ching