There are only 6 surviving lineages of mitochondrial DNA. The original group (if 5000 people were involved) probably had about 2000 females.
Or, you could have had 20,000 females, some bad luck, and only 6 lineages surviving.
Obviously all the blue-eyed blonds died out eh~!