You have a point, although I found I was always able to add enough layers to stay warm, but you can never shed enough layers to stay cool. Although it is hard to handle a fork when you have mittens on over your gloves.
Depends on what your definition of "cool enough" is. :^)
I grew up in South Louisiana, in the "pre-air conditioning" days. Summers 100F and 100% humidity. Peeled down to skivvies always did it well enough for us.
"Although it is hard to handle a fork when you have mittens on over your gloves."
Now THERE's a drawback I hadn't considered--LOL. But true.