If that were me.
“Oh, in that case I’ll go somewhere else.”
and leave.
The time to check apparel for appropriateness is upon greeting by the hostess or host at the door. If the man put his beanie on after being seated because he got a chill, it’s a little more understandable but they were still idiotic, no clue that they were embarrassing themselves and Morton’s, whether he was being truthful or not. Asking for a doctor’s note, what, is this a preschool?