Skip to comments.Random Observations: What looks like skill is often plain old luck.
Posted on 06/23/2004 6:31:09 AM PDT by xsysmgr
Heard the one about the monkey and the typewriter?
If one puts an infinite number of monkeys in front of (strongly built) typewriters and lets them clap away, there is a certainty that one of them [will] come out with an exact version of the Iliad, writes Nassim Nicholas Taleb in a recent book, Fooled by Randomness.
The monkey typist story is an old one, and the key word is infinite. But Taleb takes this hoary tale a step further. Now that we have found that hero among monkeys, would any reader invest his lifes savings on a bet that the monkey would write the Odyssey next?
Talebs point is that the past frequently tells us nothing at all about the future, even though many of us believe it does and make investments accordingly. Think about the monkey showing up at your door with his impressive past performance. Hey, he wrote the Iliad.
The lesson here for investors is powerful and frightening. How much can you rely on the track records of investment advisers, mutual fund managers, newspaper columnists, or even the market as a whole in making decisions about your investment portfolio? Not nearly as much as you probably think.
Talebs argument is that people are often tricked, mainly by the architecture of their own brains, into thinking that things that happen at random are actually happening by design. Adam Smith, the great Scottish economist and philosopher, wrote more than two centuries ago of the overweening conceit which the greater part of men have of their own abilities [and] their absurd presumption in their own good fortune.
Talebs book, which is full not only of infuriating meanderings and off-putting self-importance but also of extreme brilliance, changed the way I think about investing.
Fooled by Randomness is loaded with crackling little insights, but the best one is that what looks like skill is often plain old luck, so beware of investment geniuses. They will get their comeuppance, just as Solon warned. Solon was an upright ancient Greek legislator, known for speaking his mind. When King Croesus of Lydia, the richest man of his day, bragged to Solon about his wealth, Solon admonished, The uncertain future is yet to come, with all the variety of future. And it did. Cyrus defeated Croesus and nearly burned him at the stake.
When people buy stocks, wrote Meir Statman, a finance professor at Santa Clara University and one of the leading experts on markets, they think they are playing a game of skill. When the stock goes down rather than up, they think they have lost their knack. But they should take heart. All they have lost is luck. And next time, when the stock goes up, they should remember that was luck, too.
My own view is that its not all luck, but its mainly luck. Much of what investors do in picking stocks the research, the listening, the talking, the reading is nothing more than wheel-spinning. It wastes time and gets them nowhere special.
For example, investors and analysts are obsessed with reading tea leaves; that is, they perceive patterns that appear compelling but are actually meaningless. Burton Malkiel, the Princeton economist, disputed the value of technical analysis trying to determine where the price of a stock will go in the future based on a graph of where its been in the past in his classic book A Random Walk Down Wall Street.
He generated graphs based on the results of coin flips and showed how they looked like the head and shoulders patterns and other fetishes of chartists, as such analysts are called. Academic research has judged technical analysis useless.
And thats not all. Writes Malkiel: The academic community has rendered its judgment. Fundamental analysis is no better than technical analysis in enabling investors to capture above-average returns. To dispute fundamental analysis the examination of financial records, sales reports, macroeconomic forecasts, and the like is more disturbing. Its what conscientious investors do.
Malkiel, however, shows clearly that the value of stocks in the future often depends on unknowable events. Consider the relatively stable utilities industry. In the 1970s, utilities were deeply affected by suddenly higher oil prices; in the 1980s, by the Three Mile Island nuclear incident; in the 1990s, by deregulation; in the 2000s, by the Enron scandal.
Analysts failed to predict these random events (thats why theyre random), and earnings for the companies suffered, wrote Richard McCaffery on the Motley Fool Web site (www.fool.com). Random events are an intricate part of life in the business world, and these events make it very difficult for investors, whether professionals or not, to predict earnings and find winning investments.
Many investors and business executives who think they are geniuses are merely the beneficiaries of good fortune. We tend to think that traders are successful because they are good, Taleb writes. But the truth is that one can make money in the financial markets totally out of randomness.
Taleb describes survivorship bias a trap for investors this way: Consider 10,000 investment managers whose success depends completely on luck. Each year, half will be winners and half losers, based simply on the flip of a coin. After the first year, the 5,000 losers are fired; ditto, each succeeding year. After five years, only 313 winners, each with an impressive streak, remain.
Imagine, writes Taleb, how each of these winners would be lauded for his remarkable style, his incisive mind. . . . Some analysts may attribute his success to precise elements among his childhood experiences.
Then assume that in the following year, hes a loser. The journalists and analysts will now start laying blame, finding fault with the relaxation in his work ethic, or his dissipated lifestyle. They will find something he did before when he was successful, and attribute his failure to that. The truth will be, however, that he simply ran out of luck.
But the winners in these games figure they are so smart that they can repeat their triumphs, often with other peoples money. This disease confusing luck with skill afflicts Wall Street stock, bond, and derivatives traders, who, in the jargon of the trade, blow up with regularity after they have made a bundle.
The best example, of course, is the hedge fund Long-Term Capital Management, which was guided by the theories of two Nobel Prize economists and whose blowup in 1998 nearly brought the global financial system down with it.
So what does all this mean in practical terms? Can the average investor distinguish between luck and skill? Probably not. The lessons I draw from Taleb are: 1) If youre doing well in the market, dont get carried away by hubris. 2) Dont be reluctant to invest purely by instinct since fundamental analysis is not all its cracked up to be. 3) Pay little attention to the day-to-day movements of stocks and news about companies. 4) Dont expect mutual funds to outperform their peers simply because they have done well in the recent past. 5) Put money in low-cost index funds or broadly diversified portfolios. 6) Beware of black swans.
Taleb paraphrases the Scottish philosopher David Hume: No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion.
In other words, surprises happen. The fact that something hasnt occurred in the past such as a single-day decline of 23 percent in the Dow Jones industrial average (unknown prior to Oct. 19, 1987) or the destruction of the two tallest buildings in Manhattan (unknown prior to Sept. 11, 2001) doesnt mean it cant occur.
Heres an example of flawed reasoning of the sort that many investors practice reduced by Taleb to absurdity: I have just completed a thorough statistical examination of the life of President Bush. For 58 years, close to 21,000 observations, he did not die once. I can hence pronounce him as immortal, with a high degree of statistical significance. You may believe that just because a company has increased its profits for 10 straight years, it will keep doing so. Dont believe it.
Taleb actually makes his living as a trader pursuing black swans. He believes that, since most traders dont think black swans exist, he can get attractive odds on bets that they do.
In fact, Fooled by Randomness not only persuaded me to be aware of black swans, it encouraged me to search for them.
Which companies are being shunned by investors as having practically no chance at all for advancement? You can find such a list every week in the Value Line Investment Survey, under the heading Bargain Basement Stocks, with price-to-earnings (P/E) ratios and price-to-net-working-capital ratios that are in the bottom quartile of the research firms universe. (To find net working capital, subtract all liabilities, including long-term debt and preferred stock, from current assets.) The June 11 list comprises just 28 companies, or about 1 percent of the stocks that Value Line covers. It includes seven home builders, a sector of which I am particularly fond right now, with such names as Toll Brothers Inc. (TOL), KB Home (KBH), and Pulte Homes Inc. (PHM). Also on the list: Borders Group Inc. (BGP), bookstores; Reebok International Ltd. (RBK), shoes; Bear Stearns & Co. (BSC), securities brokerage; Brown Shoe Co. (BWS); and Lawson Products Inc. (LAWS), metal fabricating.
The theme of the book, however, is not developing stock-picking ability but cultivating a frame of mind that appreciates the role of luck and the propensity of all of us to confuse fortune with reason.
I reckon, Taleb writes, that I am not immune to such an emotional defect. But I deal with it by having no access to information, except in rare circumstances. Instead, I read poetry. Taleb has distanced himself from other members of the business community, mostly other investors and traders for whom I am developing more and more contempt.
Taleb, nevertheless, is a trader. Hes also a fellow at the Courant Institute of Mathematical Sciences at New York University, where he teaches a course on the failure of models. And he has an MBA from Wharton and a PhD from the University of Paris. His field is skeptical empiricism. He casts doubt on the things most people think they know for certain.
While skepticism is necessary for successful investing and for a successful life it can go too far. Peter L. Bernstein, in another brilliant book, Against the Gods (1996), a history of risk, quotes the Victorian writer and novelist G.K. Chesterton:
The real trouble with this world of ours is not that it is an unreasonable world, nor even that it is a reasonable one. The commonest kind of trouble is that it is nearly reasonable, but not quite. Life is not an illogicality; yet it is a trap for logicians. It looks just a little more mathematical and regular than it is; its exactitude is obvious, but its inexactitude is hidden; its wildness lies in wait.
Exactly. Investing, like life, is both random and logical, and it is excruciatingly difficult to separate the two.
James K. Glassman is a fellow at the American Enterprise Institute and host of TechCentralStation.com. He is also a member of Intel Corp.s policy advisory board. This article originally appeared in the Washington Post.
Talebs argument is that people are often tricked, mainly by the architecture of their own brains, into thinking that things that happen at random are actually happening by design. Adam Smith, the great Scottish economist and philosopher, wrote more than two centuries ago of the overweening conceit which the greater part of men have of their own abilities [and] their absurd presumption in their own good fortune.>>>
This book may be about 'investing' but it really seems to be about religious faith. After all, we are all the end result of 'incredibly unlikely randomness': one recalls the old Bloom County cartoon where Bill The Cat sits around in his underwear saying "ooooooooooohh!" as he contemplates the wonderful unlikeliness of his own existence.
Fact is, either we live in a universe where things happen for a REASON or they happen at random. If they happen at random, I'm going to spend the rest of my life in orgies then commit suicide as soon as the ol' body begins to give out.... after all, what's the difference?
Of course, if I do that, and the Universe is really a place where things happen for a REASON, I'm in seeeeerious trouble.
On second thought I guess I'll put off that tour of Plato's Retreat....
While I agree with the premis, I would argue that it is chaos, not entropy.
The author's got this backwards, but I knew what he meant.
Good post; good analysis by Mr. Glassman. I think he's right. What he is saying is that it is good to be knowledgeable about stocks, to understand market trends or seeming trends -- don't count on these: Stick to your instincts. Buy stock you believe in. Sometimes you can make a good guess in prediction. But no matter, trust your instincts.
The premise is that a group will always predict outcomes better than any single expert. He says stock analysts do not perform much better than random choices, that groups of people from diverse backgrounds are superior to any single expert in making corporate decisions, and that beginners' luck has a significant role in problem solving. (I haven't finished the book so there's probably more.)
He takes a stab at why "experts" often make bad decisions; it's because they tend to follow accepted practices within their profession. A conventional failure, he states, is accepted more easily than an unconventional success. He also addresses Trends and fads like the information boom and bust.
I started reading this book half expecting that it would be a socialist screed playing to leftist populist conceits. Surprisingly, so far its anything but that; it affirms democratic institutions as more efficient and accurate, given a minimum level of fluency. The last point is important. Taken together this affirms conservative principle.
You've got to be careful about that, too: how does one select among crowds to see which are successful, and which are not? If you pick only the "successful crowds," then the comparison is a priori biased, and likely invalid.
Friedrich Hayek predicted the fall of Communism based on the premise that a few people couldn't even access, much less properly process, the amount of information inherent in an economy.
His big insight was that the free market works better, because it allows huge numbers of people to make small decisions on things they know about, and they don't have to worry about everything else. The net effect over millions of such people and decisions is that the total information in the economy is processed more optimally. That doesn't mean that lots of people don't make bad decisions (they do). It's just that more people are able to hold enough information in their heads to make small decisions, than they are for large decisions.
Smarter people, though, can make larger decisions more accurately, which is probably why there's a such a strong correlation between IQ and income.
However, individuals are less likely to make good decisions over a broad range of fields, because they don't have enough information to do so. That's where the run-of-the-mill stock analyst resides -- he doesn't really know about the business behind a stock, he just knows the transaction side of the business, and perhaps some of the short-term business aspects, but as a generalist across many, many businesses he doesn't have much more in the way real insights on the performance of individual stocks than your average person.
Moreover, there is a certain randomness to the stock business, because there is a strong element of randomness in other peoples' investment decisions. For some reason the idea of market fluctuations reminds me of how passengers' heads all move the same way when an airliner hits turbulence. Nobody's immune to the short-term random fluctuations -- it's more a question of what happens over the course of time.
The problem with the premise of this article is that it assumes that there is no way to gain that expertise. However, there are investment companies that historically do very well -- precisely because they can afford to hire people to understand small sectors of the investment world. Those companies are just as subject to random market fluctuations as anybody else, but over the long term, they've shown the ability to pick investments that can ride through the fluctuations.
It seems to me that there's little more to this article than the seed of a sound business strategy (which has already been discovered many times).
This is generally false unless the group has a clue what's going on. A group of non-medical people will do poorly in diagnosing a disease, for example. Likewise when about 50% of the people in the US do not know that the Earth goes around the Sun in once per year, such a crowd is poor at doing most scientific stuff.
In market analysis, the crown is probably fairly knowledgeable relative to the expert anyway. This is one reason for peer-review and for having committees. It's true that a single expert can miss things that a group (by virture of its varying experience) will detect.
Game theory bump.
That advice is about the same as throwing your money away in the casinos and slots. It is not "investing" it is hoping for blindman's luck -- and taxing Providence to do so.
Investing in either "Index funds" or "broadly diversified portfolios" means you personally have added no value, no benefit to the rest of us. And you do do encourage the exploiter, the fraud, the schemer, the con man, the cheat and the sly thief -- because dang it all -- they know to go were the easy money is, were the marks and suckers are.
Your post shows a lack of knowledge of what index funds are. If you invest in a fund based on the S&P index, and the S&P goes up, you make money, if it goes down, you lose money. But the fund simply invests in all the stocks of the S&P, so you are buying those stocks. The fees and commissions are often lower for this type of investment and all that is required is to get in and out of the market at the right times. (Or hold the investment long term and hope for the average of around 10%). Either way, you do add value to the market by buying and holding the shares of the companies represented by the S&P.
As far as con men, or sharks, these are people who hope you give your money to them to invest, this can be risky if the manager you give your money to is not all that good, what is just what the article is saying.
I know what index funds are, and how they work. I've worked in the fund business, I own some funds -- not index funds, though.
OK, your comments may apply better to managed funds, especially those under not so great management.
The availibility of stupid money in large quantities -- what does it do? Of course it attracts the sleaziest, the most dishonest, the schemer and the thief.
And worse -- it takes those who could work and be productive but who are drawn by the too easy "wealth", or in the worst case, as it ruins really genius and gifted people for they are embarrassed by not being as "wealthy" as their true and accurate understandings of real talent should make them -- and by fear of embarassment of being not wealthy they chase the easy money -- and by just like chasing easy women, chasing easy money ruins many a good man.
Thought y'all might enjoy this, as it has some bearing on a mutual subject of interest.
True. People's brains are configured in a way that drives them to look for patterns among the millions of events, large and small, that they encounter each day. The survival benefit of this is obvious -- learning how the world works allows you to anticipate trouble and avoid it, spot advantageous circumstances and exploit them, and predict the results of potential actions and then implement them.
(Most animal's brains can do the same thing for the same reason, just not as well as we can, since that's our superb specialty -- anyone who owns a dog can attest at how good they are at learning the tell-tale signs (e.g. patterns) which alert them to potential food, dangers, or rides in the car.)
However, because we are driven to hunt for patterns in events, we also tend to see significance even "patterns" that aren't actually there, or trust analogies even when the events being compared are only superficially similar.
This is responsible for a great deal of "what people believe that ain't actually so", and is a constant source of study among those who take an interest in epistemology ("a branch of philosophy that investigates the origin, nature, methods, and limits of human knowledge"), such as in the magazine The Skeptical Inquirer.
And as the article at the top of this thread points out, people have a strong tendency to attribute to "skill" or "planning" even those things which simply fall into the category of "s*** happens".
This reminds me of one of the funniest quotes I've run across in a long while:
"We've heard that a million monkeys at a million keyboards could produce the Complete Works of Shakespeare; now, thanks to the Internet, we know this is not true."
- Robert Wilensky, University of California
Information theory bump.
Thanks for the ping. :-)
Indeed it does. But it's probably not a ping list thread. (At least, the odds are against it.)
I wouldn't think so, but this is definitely a bookmarkable article for support in subsequent debates.
I found an online version here: http://www.litrix.com/madraven/madne001.htm
I especially enjoyed the chapter on "Tulipomania". Wow, talk about irrational exuberance.
Beware the "rare event".
This is generally false unless the group has a clue what's going on.
You got me. I misrepresented the premise. I should have said "usually," and I did not explain, as the author did, that the groups had to have some understanding of the problem or question at hand. He did not claim that any group will always make better decisions than any single expert. I think I posted before I'd gotten my second cup. It's pretty obvious that market analysis would find polling data useful, but many of the examples cited had to do with predictions regarding concrete, verifiable facts: locating a sunken submarine, or guessing the dressed weight of a steer in a county fair contest. By averaging the best guesses of a number of experts, a point was chosen on a map that proved to be within 450 yards of where the sub was finally located. In the second example, the average of the attendees' guesses was more accurate than the best estimate of any individual expert.
In other words, I guess, more heads are better than one, given a minimal degree of expertise.
He also points out that professional groups often inhibit decisions and undermine innovation. What he termed "group think" or conformity to professional standards is behind this. I suspect that this is true, that education is only partly about knowledge; it also involves indoctrination of conventions and values. I agree with shared values and conventions if they're good and sensible. But it strikes me that a particular kind of values has taken hold.
My experience (in the mathematical and physical sciences) is that what the author terms "group think" doesn't exist. The problem isn't so much inhibiting ideas as in culling out all the bad ones. I have a pretty good reputation for making good mathematical guesses mostly because I don't often mention the 99.44% failures that I have already screened out. On the other hand, in the sciences, one always has the opportuinty of demonstrating that the ideas are good (by submitting them to peer challenges.)
I do find that politicians do seem to get caught in a stereotyped way of thinking. Perhaps it's becaust they have to please the crowd.
No doubt true, and I don't know whether or not the examples he cites can stand up to that kind of scrutiny. I've really just started the book.
But it is a perspective I never expected I'd find myself agreeing with. I, like everyone else here I imagine, have always been skeptical of the prevailing attitudes. I'm one who tends to feel reassured whenever I find myself holding the minority view -- the rugged individualism ethos. I'm sure I learned from my old man. (My cousin once said "well, your dad would argue with a stump.") And I still think it is what makes this society tick.
This author presents his ideas much better than I can here, and though I can't square all of them with what I now hold, I don't think they necessarily contradict them either. And, he ratifies what we sometimes forget is at the core of our beliefs: about representative government, free markets, equality under law, and trial by juries, etc. One thing is true; people are likely know less than they think they know. That's seems like good medicine and worth the price. I think the book is worth a read.