The best hypothesis I've seen so far on that question is in the book "Sparse Distributed Memory" by Pentti Kanerva. It proposes and mathematically analyzes a memory model that is elegant in its simplicity, but exhibits a surprising number of traits in common with what we know about human memory, including:
1. The more often it is exposed to a particular piece of data, the more accurately it recalls it.
2. The more recently it has stored a piece of data, the more accurately it recalls it.
3. Older, less reinforced "memories" tend to become more vague and generalized as more information is stored.
4. Memories are most easily retrieved when a similar (or related) piece of data is used as a "key" (i.e., the memory module is "associative").
5. The memory module generalizes when similar but somewhat differing pieces of data are stored (i.e. it figures out what they have in common, and retrieves a composite "answer" when queried with a particular).
6. It can have the "tip of the tongue" syndrome, where it knows it knows something, but can't retrieve it until a triggering memory is accessed which releases it via association.
7. It has effectively unlimited storage ability. It never simply "runs out" of data storage. Instead, if too much information is stored relative to its capacity, older memories just start degrading in quality instead of disappear altogether, and new memories can always be stored.
8. A particular piece of data is stored in no one place, but instead "all over" in the memory module. If part of the memory module is removed or damaged, it is unlikely that any particular memory will simply vanish, instead the accuracy of stored memories will somewhat degrade but most will still be retrievable. This is strikingly similar to how human memory and cognition is affected by localized brain damage.
9. The fundamental unit of the memory module is extremely simple, and the sort of thing that biological systems could easily have stumbled upon during evolution, and yet the properties of the memory model are extremely rich and powerful (including not just data storage, but many of the processes that we think of as cognition and learning).
10. The power and accuracy of the memory/cognition abilities increase simply by adding more and more of the basic "memory cells" in a repetitive way, connected literally randomly.
11. The behavior of a single memory unit is exactly what is seen in animal neurons (repeated activation lowers the threshold for subsequent activations, fewer activations raise the threshold for subsequent activations).
12. In the appendix of the book Kanerva shows that the neural structure of the human cerebellum (which is used for "muscle memory" -- e.g. learning to ride a bicycle) is strikingly similar to the proposed memory model.
13. The memory model is equally adept at storing static memories as time-sequential memories (like speeches or songs).
14. Retrieving a latter part of a time-sequential memory (like a song) is most easily achieved by using earlier parts as a "key" (i.e., like having to sing a song to yourself in order to remember a later verse).
And so on. Kanerva's memory model is amazingly versatile, and behaves very much like human memory. I think he's definitely on to something. His book is one of the most fascinating things I've ever read.
Research continues into variations on Kanerva's basic insight, such as for example this paper. Or do a web search for "sparse distributed memory", "SDM", or "kanerva memory".