Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets (Incerto)
Nassim Taleb

Ended: Nov. 26, 2012

In addition we seem to have evidence that what is called “courage” comes from an underestimation of the share of randomness in things rather than the more noble ability to stick one’s neck out for a given belief. In my experience (and in the scientific literature), economic “risk takers” are rather the victims of delusions (leading to overoptimism and overconfidence with their underestimation of possible adverse outcomes) than the opposite. Their “risk taking” is frequently randomness foolishness.
On the other hand there is the Tragic Vision of humankind that believes in the existence of inherent limitations and flaws in the way we think and act and requires an acknowledgment of this fact as a basis for any individual and collective action. This category of people includes Karl Popper (falsificationism and distrust of intellectual “answers,” actually of anyone who is confident that he knows anything with certainty), Friedrich Hayek and Milton Friedman (suspicion of governments), Adam Smith (intention of man), Herbert Simon (bounded rationality), Amos Tversky and Daniel Kahneman (heuristics and biases), the speculator George Soros, etc. The most neglected one is the misunderstood philosopher Charles Sanders Peirce, who was born a hundred years too early (he coined the term scientific “fallibilism” in opposition to Papal infallibility). Needless to say that the ideas of this book fall squarely into the Tragic category: We are faulty and there is no need to bother trying to correct our flaws. We are so defective and so mismatched to our environment that we can just work around these flaws. I am convinced of that after spending almost all my adult and professional years in a fierce fight between my brain (not Fooled by Randomness) and my emotions (completely Fooled by Randomness) in which the only success I’ve had is in going around my emotions rather than rationalizing them. Perhaps ridding ourselves of our humanity is not in the works; we need wily tricks, not some grandiose moralizing help. As an empiricist (actually a skeptical empiricist) I despise the moralizers beyond anything on this planet: I still wonder why they blindly believe in ineffectual methods. Delivering advice assumes that our cognitive apparatus rather than our emotional machinery exerts some meaningful control over our actions. We will see how modern behavioral science shows this to be completely untrue.
that which came with the help of luck could be taken away by luck (and often rapidly and unexpectedly at that). The flipside, which deserves to be considered as well (in fact it is even more of our concern), is that things that come with little help from luck are more resistant to randomness. Solon also had the intuition of a problem that has obsessed science for the past three centuries. It is called the problem of induction. I call it in this book the black swan or the rare event. Solon even understood another linked problem, which I call the skewness issue; it does not matter how frequently something succeeds if failure is too costly to bear.
He is susceptible to conference room boredom and is incapable of talking to businessmen, particularly the run-of-the-mill variety. Nero is allergic to the vocabulary of business talk, not just on plain aesthetic grounds. Phrases like “game plan,” “bottom line,” “how to get there from here,” “we provide our clients with solutions,” “our mission,” and other hackneyed expressions that dominate meetings lack both the precision and the coloration that he prefers to hear. Whether people populate silence with hollow sentences, or if such meetings present any true merit, he does not know; at any rate he did not want to be part of it. Indeed Nero’s extensive social life includes almost no businesspeople. But unlike me (I can be extremely humiliating when someone rubs me the wrong way with inelegant pompousness), Nero handles himself with gentle aloof-ness in these circumstances.
Nero believes that risk-conscious hard work and discipline can lead someone to achieve a comfortable life with a very high probability. Beyond that, it is all randomness: either by taking enormous (and unconscious) risks, or by being extraordinarily lucky. Mild success can be explainable by skills and labor. Wild success is attributable to variance.
A word on the display of emotions. Almost no one can conceal his emotions. Behavioral scientists believe that one of the main reasons why people become leaders is not from what skills they seem to possess, but rather from what extremely superficial impression they make on others through hardly perceptible physical signals—what we call today “charisma,” for example. The biology of the phenomenon is now well studied under the subject heading “social emotions.” Meanwhile some historian will “explain” the success in terms of, perhaps, tactical skills, the right education, or some other theoretical reason seen in hindsight. In addition, there seems to be curious evidence of a link between leadership and a form of psychopathology (the sociopath) that encourages the non-blinking, self-confident, insensitive person to rally followers.
Recall that Nero can be considered prosperous but not “very rich” by his day’s standards. However, according to some strange accounting measure we will see in the next chapter, he is extremely rich on the average of lives he could have led—he takes so little risk in his trading career that there could have been very few disastrous outcomes. The fact that he did not experience John’s success was the reason he did not suffer his downfall. He would be therefore wealthy according to this unusual (and probabilistic) method of accounting for wealth. Recall that Nero protects himself from the rare event. Had Nero had to relive his professional life a few million times, very few sample paths would be marred by bad luck—but, owing to his conservatism, very few as well would be affected by extreme good luck. That is, his life in stability would be similar to that of an ecclesiastic clock repairman. Naturally, we are discussing only his professional life, excluding his (sometimes volatile) private one. Arguably, in expectation, a dentist is considerably richer than the rock musician who is driven in a pink Rolls Royce, the speculator who bids up the price of impressionist paintings, or the entrepreneur who collects private jets. For one cannot consider a profession without taking into account the average of the people who enter it, not the sample of those who have succeeded in it.
start with the platitude that one cannot judge a performance in any given field (war, politics, medicine, investments) by the results, but by the costs of the alternative (i.e., if history played out in a different way). Such substitute courses of events are called alternative histories. Clearly, the quality of a decision cannot be solely judged based on its outcome, but such a point seems to be voiced only by people who fail (those who succeed attribute their success to the quality of their decision). Such opinion—“that I followed the best course”—is what politicians on their way out of office keep telling those members of the press who still listen to them—eliciting the customary commiserating “yes, we know” that makes the sting even worse. And like many platitudes, this one, while being too obvious, is not easy to carry out in practice.
I thus view people distributed across two polar categories: On one extreme, those who never accept the notion of randomness; on the other, those who are tortured by it. When I started on Wall Street in the 1980s, trading rooms were populated with people with a “business orientation,” that is, generally devoid of any introspection, flat as a pancake, and likely to be fooled by randomness. Their failure rate was extremely high, particularly when financial instruments gained in complexity. Somehow, tricky products, like exotic options, were introduced and carried counterintuitive payoffs that were too difficult for someone of such culture to handle. They dropped like flies; I do not think that many of the hundreds of MBAs of my generation I met on Wall Street in the 1980s still engage in such forms of professional and disciplined risk taking.
When MBAs apply for trading positions, they frequently boast “advanced” chess skills on their résumés. I recall the MBA career counselor at Wharton recommending our advertising chess skills “because it sounds intelligent and strategic.” MBAs, typically, can interpret their superficial knowledge of the rules of the game into “expertise.” We used to verify the accuracy of claims of chess expertise (and the character of the applicant) by pulling a chess set out of a drawer and telling the student, now turning pale: “Yuri will have a word with you.”
The interview is illustrative of the destructive aspect of the media, in catering to our heavily warped common sense and biases. I was told that George Will was very famous and extremely respected (that is, for a journalist). He might even be someone of the utmost intellectual integrity; his profession, however, is merely to sound smart and intelligent to the hordes. Shiller, on the other hand, understands the ins and outs of randomness; he is trained to deal with rigorous argumentation, but does sound less smart in public because his subject matter is highly counterintuitive. Shiller had been pronouncing the stock market to be overpriced for a long time. George Will indicated to Shiller that had people listened to him in the past they would have lost money, as the market has more than doubled since he started pronouncing it overvalued. To such a journalistic and well-sounding (but senseless) argument, Shiller was unable to respond except to explain that the fact that he was wrong in one single market call should not carry undue significance. Shiller, as a scientist, did not claim to be a prophet or one of the entertainers who comment on the markets on the evening news. Yogi Berra would have had a better time with his confident comment on the fat lady not having sung yet.
As a derivatives trader I noticed that people do not like to insure against something abstract; the risk that merits their attention is always something vivid.
In that sense the description coming from journalism is certainly not just an unrealistic representation of the world but rather the one that can fool you the most by grabbing your attention via your emotional apparatus—the cheapest to deliver sensation. Take the mad cow “threat” for example: Over a decade of hype, it only killed people (in the highest estimates) in the hundreds as compared to car accidents (several hundred thousands!)—except that the journalistic description of the latter would not be commercially fruitful. (Note that the risk of dying from food poisoning or in a car accident on the way to a restaurant is greater than dying from mad cow disease.) This sensationalism can divert empathy toward wrong causes: cancer and malnutrition being the ones that suffer the most from the lack of such attention.
The dividend of the computer revolution to us did not come in the flooding of self-perpetuating e-mail messages and access to chat rooms; it was in the sudden availability of fast processors capable of generating a million sample paths per minute. Recall that I never considered myself better than an unenthusiastic equation solver and was rarely capable of prowess in the matter—being better at setting up equations than solving them. Suddenly, my engine allowed me to solve with minimal effort the most intractable of equations. Few solutions became out of reach.
As I mentioned above, it is not natural for us to learn from history. We have enough clues to believe that our human endowment does not favor transfers of experience in a cultural way but through selection of those who bear some favorable traits. It is a platitude that children learn only from their own mistakes; they will cease to touch a burning stove only when they are themselves burned; no possible warning by others can lead to developing the smallest form of cautiousness. Adults, too, suffer from such a condition. This point has been examined by behavioral economics pioneers Daniel Kahneman and Amos Tversky with regard to the choices people make in selecting risky medical treatments—I myself have seen it in my being extremely lax in the area of detection and prevention (i.e., I refuse to derive my risks from the probabilities computed on others, feeling that I am somewhat special) yet extremely aggressive in the treatment of medical conditions (I overreact when I am burned), which is not coherent with rational behavior under uncertainty. This congenital denigration of the experience of others is not limited to children or to people like myself; it affects business decision makers and investors on a grand scale.
Since then plenty of discussions of amnesic patients show some form of learning on the part of people without their being aware of it and without it being stored in conscious memory. The scientific name of the distinction between the two memories, the conscious and the nonconscious, is declarative and nondeclarative. Much of the risk avoidance that comes from experiences is part of the second. The only way I developed a respect for history is by making myself aware of the fact that I was not programmed to learn from it in a textbook format.
Actually, things can be worse than that: In some respects we do not learn from our own history. Several branches of research have been examining our inability to learn from our own reactions to past events: For example, people fail to learn that their emotional reactions to past experiences (positive or negative) were short-lived—yet they continuously retain the bias of thinking that the purchase of an object will bring long-lasting, possibly permanent, happiness or that a setback will cause severe and prolonged distress (when in the past similar setbacks did not affect them for very long and the joy of the purchase was short-lived).
Things are always obvious after the fact. The civil servant was a very intelligent person, and this mistake is much more prevalent than one would think. It has to do with the way our mind handles historical information. When you look at the past, the past will always be deterministic, since only one single observation took place. Our mind will interpret most events not with the preceding ones in mind, but the following ones. Imagine taking a test knowing the answer. While we know that history flows forward, it is difficult to realize that we envision it backward. Why is it so? We will discuss the point in Chapter 11 but here is a possible explanation: Our minds are not quite designed to understand how the world works, but, rather, to get out of trouble rapidly and have progeny. If they were made for us to understand things, then we would have a machine in it that would run the past history as in a VCR, with a correct chronology, and it would slow us down so much that we would have trouble operating. Psychologists call this overestimation of what one knew at the time of the event due to subsequent information the hindsight bias, the “I knew it all along” effect. Now the civil servant called the trades that ended up as losers “gross mistakes,” just like journalists call decisions that end up costing a candidate his election a “mistake.” I will repeat this point until I get hoarse: A mistake is not something to be determined after the fact, but in the light of the information until that point.
There is an important and nontrivial aspect of historical thinking, perhaps more applicable to the markets than anything else: Unlike many “hard” sciences, history cannot lend itself to experimentation. But somehow, overall, history is potent enough to deliver, on time, in the medium to long run, most of the possible scenarios, and to eventually bury the bad guy. Bad trades catch up with you, it is frequently said in the markets. Mathematicians of probability give that a fancy name: ergodicity. It means, roughly, that (under certain conditions) very long sample paths would end up resembling each other. The properties of a very, very long sample path would be similar to the Monte Carlo properties of an average of shorter ones. The janitor in Chapter 1 who won the lottery, if he lived one thousand years, cannot be expected to win more lotteries. Those who were unlucky in life in spite of their skills would eventually rise. The lucky fool might have benefited from some luck in life; over the longer run he would slowly converge to the state of a less-lucky idiot. Each one would revert to his long-term properties.
In the next step I will show how my Monte Carlo toy taught me to favor distilled thinking, by which I mean the thinking based on information around us that is stripped of meaningless but diverting clutter.
For an idea to have survived so long across so many cycles is indicative of its relative fitness. Noise, at least some noise, was filtered out. Mathematically, progress means that some new information is better than past information, not that the average of new information will supplant past information, which means that it is optimal for someone, when in doubt, to systematically reject the new idea, information, or method. Clearly and shockingly, always. Why? The argument in favor of “new things” and even more “new new things” goes as follows: Look at the dramatic changes that have been brought about by the arrival of new technologies, such as the automobile, the airplane, the telephone, and the personal computer. Middlebrow inference (inference stripped of probabilistic thinking) would lead one to believe that all new technologies and inventions would likewise revolutionize our lives. But the answer is not so obvious: Here we only see and count the winners, to the exclusion of the losers (it is like saying that actors and writers are rich, ignoring the fact that actors are largely waiters—and lucky to be ones, for the less comely writers usually serve French fries at McDonald’s). Losers? The Saturday newspaper lists dozens of new patents of such items that can revolutionize our lives. People tend to infer that because some inventions have revolutionized our lives that inventions are good to endorse and we should favor the new over the old. I hold the opposite view. The opportunity cost of missing a “new new thing” like the airplane and the automobile is minuscule compared to the toxicity of all the garbage one has to go through to get to these jewels (assuming these have brought some improvement to our lives, which I frequently doubt). Now the exact same argument applies to information. The problem with information is not that it is diverting and generally useless, but that it is toxic. We will examine the dubious value of the highly frequent news with a more technical discussion of signal filtering and observation frequency farther down. I will say here that such respect for the time-honored provides arguments to rule out any commerce with the babbling modern journalist and implies a minimal exposure to the media as a guiding principle for someone involved in decision making under uncertainty. If there is anything better than noise in the mass of “urgent” news pounding us, it would be like a needle in a haystack. People do not realize that the media is paid to get your attention. For a journalist, silence rarely surpasses any word.
Things are not getting any better these days. At the time of writing, news providers are offering all manner of updates, “breaking news” that can be delivered electronically in a wireless manner. The ratio of undistilled information to distilled is rising, saturating markets. The elder’s messages need not be delivered to you as imminent news. This does not mean that all journalists are fooled by randomness noise providers: There are hordes of thoughtful journalists in the business (I would suggest London’s Anatole Kaletsky and New York’s Jim Grant and Alan Abelson as the underrated representatives of such a class among financial journalists; Gary Stix among scientific journalists); it is just that prominent media journalism is a thoughtless process of providing the noise that can capture people’s attention and there exists no mechanism for separating the two. As a matter of fact, smart journalists are often penalized. Like the lawyer in Chapter 11 who does not care about the truth, but about arguments that can sway a jury whose intellectual defects he knows intimately, journalism goes to what can capture our attention, with adequate sound bites. Again, my scholarly friends would wonder why I am getting emotional stating the obvious things about the journalists; the problem with my profession is that we depend on them for what information we need to obtain.
I was amused to discover a similar evolutionary argument in mate selection that considers that women prefer (on balance) to mate with healthy older men over healthy younger ones, everything else being equal, as the former provide some evidence of better genes. Gray hair signals an enhanced ability to survive—conditional on having reached the gray hair stage, a man is likely to be more resistant to the vagaries of life.
Let us manufacture a happily retired dentist, living in a pleasant, sunny town. We know a priori that he is an excellent investor, and that he will be expected to earn a return of 15% in excess of Treasury bills, with a 10% error rate per annum (what we call volatility). It means that out of 100 sample paths, we expect close to 68 of them to fall within a band of plus and minus 10% around the 15% excess return, i.e., between 5% and 25% (to be technical; the bell-shaped normal distribution has 68% of all observations falling between -1 and 1 standard deviations). It also means that 95 sample paths would fall between -5% and 35%. Clearly, we are dealing with a very optimistic situation. The dentist builds for himself a nice trading desk in his attic, aiming to spend every business day there watching the market, while sipping decaffeinated cappuccino. He has an adventurous temperament, so he finds this activity more attractive than drilling the teeth of reluctant little old Park Avenue ladies. He subscribes to a Web-based service that supplies him with continuous prices, now to be obtained for a fraction of what he pays for his coffee. He puts his inventory of securities in his spreadsheet and can thus instantaneously monitor the value of his speculative portfolio. We are living in the era of connectivity. A 15% return with a 10% volatility (or uncertainty) per annum translates into a 93% probability of success in any given year. But seen at a narrow time scale, this translates into a mere 50.02% probability of success over any given second as shown in Table 3.1. Over the very narrow time increment, the observation will reveal close to nothing. Yet the dentist’s heart will not tell him that. Being emotional, he feels a pang with every loss, as it shows in red on his screen. He feels some pleasure when the performance is positive, but not in equivalent amount as the pain experienced when the performance is negative. Table 3.1 Probability of success at different scales Scale                                               Probability 1 year                                                                          93% 1 quarter                                                                          77% 1 month                                                                          67% 1 day                                               54% 1 hour                                               51.3% 1 minute                                               50.17% 1 second                                               50.02% At the end of every day the dentist will be emotionally drained. A minute-by-minute examination of his performance means that each day (assuming eight hours per day) he will have 241 pleasurable minutes against 239 unpleasurable ones. These amount to 60,688 and 60,271, respectively, per year. Now realize that if the unpleasurable minute is worse in reverse pleasure than the pleasurable minute is in pleasure terms, then the dentist incurs…
Over a short time increment, one observes the variability of the portfolio, not the returns. In other words, one sees the variance, little else. I always remind myself that what one observes is at best a combination of variance and returns, not just returns (but my emotions do not care about what I tell myself). 2.         Our emotions are not designed to understand the point. The dentist did better when he dealt with monthly statements rather than more frequent ones. Perhaps it would be even better for him if he limited himself to yearly statements. (If you think that you can control your emotions, think that some people also believe that they can control their heartbeat or hair growth.) 3.         When I see an investor monitoring his portfolio with live prices on his cellular telephone or his handheld, I smile and smile. Finally, I reckon that I am not immune to such an emotional defect. But I deal with it by having no access to information, except in rare circumstances. Again, I prefer to read poetry. If an event is important enough, it will find its way to my ears. I will return to this point in time.
The same methodology can explain why the news (the high scale) is full of noise and why history (the low scale) is largely stripped of it (though fraught with interpretation problems). This explains why I prefer not to read the newspaper (outside of the obituary), why I never chitchat about markets, and, when in a trading room, I frequent the mathematicians and the secretaries, not the traders. It explains why it is better to read The New Yorker on Mondays than The Wall Street Journal every morning (from the standpoint of frequency, aside from the massive gap in intellectual class between the two publications). Finally, this explains why people who look too closely at randomness burn out, their emotions drained by the series of pangs they experience. Regardless of what people claim, a negative pang is not offset by a positive one (some psychologists estimate the negative effect for an average loss to be up to 2.5 the magnitude of a positive one); it will lead to an emotional deficit.
My problem is that I am not rational and I am extremely prone to drown in randomness and to incur emotional torture. I am aware of my need to ruminate on park benches and in cafés away from information, but I can only do so if I am somewhat deprived of it. My sole advantage in life is that I know some of my weaknesses, mostly that I am incapable of taming my emotions facing news and incapable of seeing a performance with a clear head. Silence is far better.
This is even more apparent when the literary intellectual starts using scientific buzzwords, like “uncertainty principle,” “Gödel’s theorem,” “parallel universe,” or “relativity,” either out of context or, as often, in exact opposition to the scientific meaning. I suggest reading the hilarious Fashionable Nonsense by Alan Sokal for an illustration of such practice
This was one of the small points that emerging-market economists around the globe, from talking to each other so much, forgot to take into account. Veteran trader Marty O’Connell calls this the firehouse effect. He had observed that firemen with much downtime who talk to each other for too long come to agree on many things that an outside, impartial observer would find ludicrous (they develop political ideas that are very similar). Psychologists give it a fancier name, but my friend Marty has no training in behavioral sciences.
And, at any point in time, the richest traders are often the worst traders. This, I will call the cross-sectional problem: At a given time in the market, the most successful traders are likely to be those that are best fit to the latest cycle. This does not happen too often with dentists or pianists—because these professions are more immune to randomness.
An overestimation of the accuracy of their beliefs in some measure, either economic (Carlos) or statistical (John). They never considered that the fact that trading on economic variables has worked in the past may have been merely coincidental, or, perhaps even worse, that economic analysis was fit to past events to mask the random element in it. Consider that of all the possible economic theories available, one can find a plausible one that explains the past, or a portion of it.
A tendency to get married to positions. There is a saying that bad traders divorce their spouse sooner than abandon their positions. Loyalty to ideas is not a good thing for traders, scientists—or anyone.
The tendency to change their story. They become investors “for the long haul” when they are losing money, switching back and forth between traders and investors to fit recent reversals of fortune. The difference between a trader and an investor lies in the duration of the bet, and the corresponding size. There is absolutely nothing wrong with investing “for the long haul,” provided one does not mix it with short-term trading—it is just that many people become long-term investors after they lose money, postponing their decision to sell as part of their denial.
No precise game plan ahead of time as to what to do in the event of losses. They simply were not aware of such a possibility. Both bought more bonds after the market declined sharply, but not in response to a predetermined plan.
Absence of critical thinking expressed in absence of revision of their stance with “stop losses.” Middlebrow traders do not like selling when it is “even better value.” They did not consider that perhaps their method of determining value is wrong, rather than the market failing to accommodate their measure of value. They may be right, but, perhaps, some allowance for the possibility of their methods being flawed was not made. For all his flaws, we will see that Soros seems rarely to examine an unfavorable outcome without testing his own framework of analysis.
Denial. When the losses occurred there was no clear acceptance of what had happened. The price on the screen lost its reality in favor of some abstract “value.” In classic denial mode, the usual “this is only the result of liquidation, distress sales” was proffered. They continuously ignored the message from reality.
Recall that someone with only casual knowledge about the problems of randomness would believe that an animal is at the maximum fitness for the conditions of its time. This is not what evolution means; on average, animals will be fit, but not every single one of them, and not at all times. Just as an animal could have survived because its sample path was lucky, the “best” operators in a given business can come from a subset of operators who survived because of overfitness to a sample path—a sample path that was free of the evolutionary rare event. One vicious attribute is that the longer these animals can go without encountering the rare event, the more vulnerable they will be to it. We said that should one extend time to infinity, then, by ergodicity, that event will happen with certainty—the species will be wiped out! For evolution means fitness to one and only one time series, not the average of all the possible environments.
The best description of my lifelong business in the market is “skewed bets,” that is, I try to benefit from rare events, events that do not tend to repeat themselves frequently, but, accordingly, present a large payoff when they occur. I try to make money infrequently, as infrequently as possible, simply because I believe that rare events are not fairly valued, and that the rarer the event, the more undervalued it will be in price. In addition to my own empiricism, I think that the counterintuitive aspect of the trade (and the fact that our emotional wiring does not accommodate it) gives me some form of advantage. Why are these events poorly valued? Because of a psychological bias; people who surrounded me in my career were too focused on memorizing section 2 of The Wall Street Journal during their train ride to reflect properly on the attributes of random events. Or perhaps they watched too many gurus on television. Or perhaps they spent too much time upgrading their PalmPilot. Even some experienced trading veterans do not seem to get the point that frequencies do not matter. Jim Rogers, a “legendary” investor, made the following statement: I don’t buy options. Buying options is another way to go to the poorhouse. Someone did a study for the SEC and discovered that 90 percent of all options expire as losses. Well, I figured out that if 90 percent of all long option positions lost money, that meant that 90 percent of all short option positions make money. If I want to use options to be bearish, I sell calls. Visibly, the statistic that 90% of all option positions lost money is meaningless, (i.e., the frequency) if we do not take into account how much money is made on average during the remaining 10%. If we make 50 times our bet on average when the option is in the money, then I can safely make the statement that buying options is another way to go to the palazzo rather than the poorhouse. Mr. Jim Rogers seems to have gone very far in life for someone who does not distinguish between probability and expectation (strangely, he was the partner of George Soros, a complex man who thrived on rare events—more on him later).
Currencies that exhibit the largest historical stability, for example, are the most prone to crashes. This was bitterly discovered in the summer of 1997 by investors who chose the safety of the pegged currencies of Malaysia, Indonesia, and Thailand (they were pegged to the U.S. dollar in a manner to exhibit no volatility, until their sharp, sudden, and brutal devaluations). We could be either too lax or too stringent in accepting past information as a prediction of the future. As a skeptic, I reject a sole time series of the past as an indication of future performance; I need a lot more than data. My major reason is the rare event, but I have plenty of others.
Common statistical method is based on the steady augmentation of the confidence level, in nonlinear proportion to the number of observations. That is, for an n times increase in the sample size, we increase our knowledge by the square root of n. Suppose I am drawing from an urn containing red and black balls. My confidence level about the relative proportion of red and black balls after 20 drawings is not twice the one I have after 10 drawings; it is merely multiplied by the square root of 2 (that is, 1.41).
Where statistics becomes complicated, and fails us, is when we have distributions that are not symmetric, like the urn above. If there is a very small probability of finding a red ball in an urn dominated by black ones, then our knowledge about the absence of red balls will increase very slowly—more slowly than at the expected square root of n rate. On the other hand, our knowledge of the presence of red balls will dramatically improve once one of them is found. This asymmetry in knowledge is not trivial; it is central in this book—it is a central philosophical problem for such people as the ancient skeptics David Hume and Karl Popper (on that, later).
In his Treatise on Human Nature, the Scots philosopher David Hume posed the issue in the following way (as rephrased in the now famous black swan problem by John Stuart Mill): No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion. Hume had been irked by the fact that science in his day (the eighteenth century) had experienced a swing from scholasticism, entirely based on deductive reasoning (no emphasis on the obsdervation of the real world) to, owing to Francis Bacon, an overreaction into naive and unstructured empiricism. Bacon had argued against “spinning the cobweb of learning” with little practical result (science resembled theology). Science had shifted, thanks to Bacon, into an emphasis on empirical observation. The problem is that, without a proper method, empirical observations can lead you astray. Hume came to warn us against such knowledge, and to stress the need for some rigor in the gathering and interpretation of knowledge—what is called epistemology (from episteme, Greek for learning). Hume is the first modern epistemologist (epistemologists operating in the applied sciences are often called methodologists or philosophers of science). What I am writing here is not strictly true, for Hume said things far worse than that; he was an obsessive skeptic and never believed that a link between two items could be truly established as being causal. But we will tone him down a bit for this book.
At the center of his modus is Niederhoffer’s dogma that any “testable” statement should be tested, as our minds make plenty of empirical mistakes when relying on vague impressions. His advice is obvious, but it is rarely practiced. How many effects we take for granted might not be there? A testable statement is one that can be broken down into quantitative components and subjected to statistical examination. For instance, a conventional-wisdom, empirical style statement like automobile accidents happen closer to home can be tested by taking the average distance between the accident and the domicile of the driver (if, say, about 20% of accidents happen within a twelve-mile radius). However, one needs to be careful in the interpretation; a naive reader of the result would tell you that you are more likely to have an accident if you drive in your neighborhood than if you did so in remote places, which is a typical example of naive empiricism. Why? Because accidents may happen closer to home simply because people spend their time driving close to home (if people spend 20% of their time driving in a twelve-mile radius).
But there is a more severe aspect of naive empiricism. I can use data to disprove a proposition, never to prove one. I can use history to refute a conjecture, never to affirm it. For instance, the statement The market never goes down 20% in a given three-month period can be tested but is completely meaningless if verified. I can quantitatively reject the proposition by finding counterexamples, but it is not possible for me to accept it simply because, in the past, the market never went down 20% in any three-month period (you cannot easily make the logical leap from “has never gone down” to “never goes down”). Samples can be greatly insufficient; markets may change; we may not know much about the market from historical information. You can more safely use the data to reject than to confirm hypotheses. Why? Consider the following statements: Statement A: No swan is black, because I looked at four thousand swans and found none. Statement B: Not all swans are white. I cannot logically make statement A, no matter how many successive white swans I may have observed in my life and may observe in the future (except, of course, if I am given the privilege of observing with certainty all available swans). It is, however, possible to make Statement B merely by finding one single counterexample. Indeed, Statement A was disproved by the discovery of Australia, as it led to the sighting of the Cygnus atratus, a swan variety that was jet black! The reader will see a hint of Popper’s ideas, as there is a strong asymmetry between the two statements; and, furthermore, such asymmetry lies in the foundations of knowledge. It is also at the core of my operation as a decision maker under uncertainty.
I suddenly felt financially insecure and feared becoming an employee of some firm that would turn me into a corporate slave with “work ethics” (whenever I hear work ethics I interpret inefficient mediocrity).
There are only two types of theories: 1. Theories that are known to be wrong, as they were tested and adequately rejected (he calls them falsified). 2. Theories that have not yet been known to be wrong, not falsified yet, but are exposed to be proved wrong. Why is a theory never right? Because we will never know if all the swans are white (Popper borrowed the Kantian idea of the flaws in our mechanisms of perception). The testing mechanism may be faulty. However, the statement that there is a black swan is possible to make. A theory cannot be verified. To paraphrase baseball coach Yogi Berra again, past data has a lot of good in it, but it is the bad side that is bad. It can only be provisionally accepted. A theory that falls outside of these two categories is not a theory. A theory that does not present a set of conditions under which it would be considered wrong would be termed charlatanism—it-would be impossible to reject otherwise. Why? Because the astrologist can always find a reason to fit the past event, by saying that Mars was probably in line but not too much so (likewise to me a trader who does not have a point that would make him change his mind is not a trader). Indeed the difference between Newtonian physics, which was falsified by Einstein’s relativity, and astrology lies in the following irony. Newtonian physics is scientific because it allowed us to falsify it, as we know that it is wrong, while astrology is not because it does not offer conditions under which we could reject it. Astrology cannot be disproved, owing to the auxiliary hypotheses that come into play. Such point lies at the basis of the demarcation between science and nonsense (called “the problem of demarcation”).
More practically to me, Popper had many problems with statistics and statisticians. He refused to blindly accept the notion that knowledge can always increase with incremental information—which is the foundation of statistical inference. It may in some instances, but we do not know which ones. Many insightful people, such as John Maynard Keynes, independently reached the same conclusions.
The reason I feel that he is important for us traders is because to him the matter of knowledge and discovery is not so much in dealing with what we know as in dealing with what we do not know. His famous quote: These are men with bold ideas, but highly critical of their own ideas; they try to find whether their ideas are right by trying first to find whether they are not perhaps wrong. They work with bold conjectures and severe attempts at refuting their own conjectures. “These” are scientists. But they could be anything.
I am an exceedingly naive falsificationist. Why? Because I can survive being one. My extreme and obsessive Popperism is carried out as follows. I speculate in all of my activities on theories that represent some vision of the world, but with the following stipulation: No rare event should harm me. In fact, I would like all conceivable rare events to help me. My idea of science diverges with that of the people around me walking around calling themselves scientists. Science is mere speculation, mere formulation of conjecture.
I will refrain from commonplace discourse about the divorce between those who have ideas and those who carry them in practice, except to bring out the interesting behavioral problem; we like to emit logical and rational ideas but we do not necessarily enjoy this execution. Strange as it sounds, this point has only been discovered very recently (we will see that we are not genetically fit to be rational and act rationally; we are merely fit for the maximum probability of transmitting our genes in some given unsophisticated environment). Also, strange as it sounds, George Soros, obsessively self-critical, seems to be more Popperian than Popper in his professional behavior.
Memory in humans is a large machine to make inductive inferences. Think of memories: What is easier to remember, a collection of random facts glued together, or a story, something that offers a series of logical links? Causality is easier to commit to memory. Our brain would have less work to do in order to retain the information. The size is smaller. What is induction exactly? Induction is going from plenty of particulars to the general. It is very handy, as the general takes much less room in one’s memory than a collection of particulars. The effect of such compression is the reduction in the degree of detected randomness.
If the science of statistics can benefit me in anything, I will use it. If it poses a threat, then I will not. I want to take the best of what the past can give me without its dangers. Accordingly, I will use statistics and inductive methods to make aggressive bets, but I will not use them to manage my risks and exposure. Surprisingly, all the surviving traders I know seem to have done the same. They trade on ideas based on some observation (that includes past history) but, like the Popperian scientists, they make sure that the costs of being wrong are limited (and their probability is not derived from past data). Unlike Carlos and John, they know before getting involved in the trading strategy which events would prove their conjecture wrong and allow for it (recall that Carlos and John used past history both to make their bets and to measure their risk). They would then terminate their trade. This is called a stop loss, a predetermined exit point, a protection from the black swan. I find it rarely practiced.
The composition of Part I made me even more confident in my withdrawal from the media and my distancing myself from other members of the business community, mostly other investors and traders for whom I am developing more and more contempt. I believe that I cannot have power over myself as I have an ingrained desire to integrate among people and cultures and would end up resembling them; by withdrawing myself entirely I can have a better control of my fate. I am currently enjoying a thrill of the classics I have not felt since childhood. I am now thinking of the next step: to recreate a low-information, more deterministic ancient time, say in the nineteenth century, all the while benefiting from some of the technical gains (such as the Monte Carlo engine), all of the medical breakthroughs, and all the gains of social justice of our age. I would then have the best of everything. This is called evolution.
A few years ago, when I told one A., a then Master-of-the-Universe type, that track records were less relevant than he thought, he found the remark so offensive that he violently flung his cigarette lighter in my direction. The episode taught me a lot. Remember that nobody accepts randomness in his own success, only his failure. His ego was pumped up as he was heading up a department of “great traders” who were then temporarily making a fortune in the markets and attributing the idea to the soundness of their business, their insights, or their intelligence. They subsequently blew up during the harsh New York winter of 1994 (it was the bond market crash that followed the surprise interest rate hike by Alan Greenspan). The interesting part is that several years later I can hardly find any of them still trading (ergodicity).
We have so far discussed the spurious survivor—the same logic applies to the skilled person who has the odds markedly stacked in her favor, but who still ends up going to the cemetery. This effect is the exact opposite of the survivorship bias. Consider that all one needs is two bad years in the investment industry to terminate a risk-taking career and that, even with great odds in one’s favor, such an outcome is very possible. What do people do to survive? They maximize their odds of staying in the game by taking black-swan risks (like John and Carlos)—those that fare well most of the time, but incur a risk of blowing up.
A similar misconception of probabilities arises from the random encounters one may have with relatives or friends in highly unexpected places. “It’s a small world!” is often uttered with surprise. But these are not improbable occurrences—the world is much larger than we think. It is just that we are not truly testing for the odds of having an encounter with one specific person, in a specific location at a specific time. Rather, we are simply testing for any encounter, with any person we have ever met in the past, and in any place we will visit during the period concerned. The probability of the latter is considerably higher, perhaps several thousand times the magnitude of the former.
When the statistician looks at the data to test a given relationship, say, to ferret out the correlation between the occurrence of a given event, like a political announcement, and stock market volatility, odds are that the results can be taken seriously. But when one throws the computer at data, looking for just about any relationship, it is certain that a spurious connection will emerge, such as the fate of the stock market being linked to the length of women’s skirts. And just like the birthday coincidences, it will amaze people.
The same mechanism is behind the formation of conspiracy theories. Like The Bible Code they can seem perfect in their logic and can cause otherwise intelligent people to fall for them. I can create a conspiracy theory by downloading hundreds of paintings from an artist or group of artists and finding a constant among all those paintings (among the hundreds of thousand of traits). I would then concoct a conspiratorial theory around a secret message shared by these paintings. This is seemingly what the author of the bestselling The Da Vinci Code did.
My favorite time is spent in bookstores, where I aimlessly move from book to book in an attempt to make a decision as to whether to invest the time in reading it. My buying is frequently made on impulse, based on superficial but suggestive clues. Frequently, I have nothing but a book jacket as appendage to my decision making. Jackets often contain praise by someone, famous or not, or excerpts from a book review. Good praise by a famous and respected person or a well-known magazine would sway me into buying the book. What is the problem? I tend to confuse a book review, which is supposed to be an assessment of the quality of the book, with the best book reviews, marred with the same survivorship biases. I mistake the distribution of the maximum of a variable with that of the variable itself. The publisher will never put on the jacket of the book anything but the best praise. Some authors go even a step beyond, taking a tepid or even unfavorable book review and selecting words in it that appear to praise the book. One such example came from one Paul Wilmott (an English financial mathematician of rare brilliance and irreverence) who managed to announce that I gave him his “first bad review,” yet used excerpts from it as praise on the book jacket (we later became friends, which allowed me to extract an endorsement from him for this book).
To prove their claim, they present the convincing testimonial of someone who was cured thanks to their methods. For instance, I once saw a former throat cancer patient explaining how he was saved by a combination of vitamins for sale for the exceptionally low price of $14.95—in all likelihood he was sincere (although of course compensated for his account, perhaps with a lifetime supply of such medicine). In spite of our advances, people still believe in the existence of links between disease and cure based on such information, and there is no scientific evidence that can convince them more potently than a sincere and emotional testimonial. Such testimonial does not always come from the regular guy; statements by Nobel Prize winners (in the wrong discipline) could easily suffice. Linus Pauling, a Nobel Prize winner in chemistry, was said to believe in vitamin C’s medicinal properties, himself ingesting massive daily doses. With his bully pulpit, he contributed to the common belief in vitamin C’s curative properties. Many medical studies, unable to replicate Pauling’s claims, fell on deaf ears as it was difficult to undo the testimonial by a “Nobel Prize winner,” even if he was not qualified to discuss matters related to medicine.
Many of these claims have been harmless outside of the financial profits for these charlatans—but many cancer patients may have replaced the more scientifically investigated therapies, in favor of these methods, and died as a result of their neglecting more orthodox cures (again, the nonscientific methods are gathered under what is called “alternative medicine,” that is, unproven therapies, and the medical community has difficulties convincing the press that there is only one medicine and that alternative medicine is not medicine). The reader might wonder about my claims that the user of these products could be sincere, without it meaning that he was cured by the illusory treatment. The reason is something called “spontaneous remission,” in which a very small minority of cancer patients, for reasons that remain entirely speculative, wipe out cancer cells and recover “miraculously.” Some switch causes the patient’s immune system to eradicate all cancer cells from the body. These people would have been equally cured by drinking a glass of Vermont spring water or chewing on dried beef as they were by taking these beautifully wrapped pills. Finally, these spontaneous remissions might not be so spontaneous; they might, at the bottom, have a cause that we are not yet sophisticated enough to detect.
I would further illustrate the point with the study of a phenomenon well-known as cancer clusters. Consider a square with 16 random darts hitting it with equal probability of being at any place in the square. If we divide the square into 16 smaller squares, it is expected that each smaller square will contain one dart on average—but only on average. There is a very small probability of having exactly 16 darts in 16 different squares. The average grid will have more than one dart in a few squares, and no dart at all in many squares. It will be an exceptionally rare incident that no (cancer) cluster would show on the grid. Now, transpose our grid with the darts in it to overlay a map of any region. Some newspaper will declare that one of the areas (the one with more than the average of darts) harbors radiation that causes cancer, prompting lawyers to start soliciting the patients.
By the same argument, science is marred by a pernicious survivorship bias, affecting the way research gets published. In a way that is similar to journalism, research that yields no result does not make it to print. That may seem sensible, as newspapers do not have to have a screaming headline saying that nothing new is taking place (though the Bible was smart enough to declare ein chadash tachat hashemesh—“nothing new under the sun,” providing the information that things just do recur). The problem is that a finding of absence and an absence of findings get mixed together. There may be great information in the fact that nothing took place. As Sherlock Holmes noted in the Silver Blaze case—the curious thing was that the dog did not bark. More problematic, there are plenty of scientific results that are left out of publications because they are not statistically significant, but nevertheless provide information.
Outside of this very specialized bookmaker-style profession, to be honest, I am unable to answer the question of who’s lucky or unlucky. I can tell that person A seems less lucky than person B, but the confidence in such knowledge can be so weak as to be meaningless. I prefer to remain a skeptic. People frequently misinterpret my opinion. I never said that every rich man is an idiot and every unsuccessful person unlucky, only that in absence of much additional information it is preferable to reserve one’s judgment. It is safer.
Our brain is not cut out for nonlinearities. People think that if, say, two variables are causally linked, then a steady input in one variable should always yield a result in the other one. Our emotional apparatus is designed for linear causality. For instance, you study every day and learn something in proportion to your studies. If you do not feel that you are going anywhere, your emotions will cause you to become demoralized. But reality rarely gives us the privilege of a satisfying linear positive progression: You may study for a year and learn nothing, then, unless you are disheartened by the empty results and give up, something will come to you in a flash. My partner Mark Spitznagel summarizes it as follows: Imagine yourself practicing the piano every day for a long time, barely being able to perform “Chopsticks,” then suddenly finding yourself capable of playing Rachmaninov. Owing to this nonlinearity, people cannot comprehend the nature of the rare event. This summarizes why there are routes to success that are nonrandom, but few, very few, people have the mental stamina to follow them. Those who go the extra mile are rewarded. In my profession one may own a security that benefits from lower market prices, but may not react at all until some critical point. Most people give up before the rewards.
Table 11.1 Trader and Scientific Approach There
Our brain functions by “modules.” An interesting aspect of modularity is that we may use different modules for different instances of the same problem, depending on the framework in which it is presented—as discussed in the notes to this section. One of the attributes of a module is its “encapsulation,” i.e., we cannot interfere with its functioning, as we are not aware of using it. The most striking module is used when we try to find a cheater. Expressed in purely logical form (though with extreme clarity), a given quiz is only solved by 15% of the people to whom it is given. Now, the same quiz expressed in a manner that aims at uncovering a cheater, almost everyone gets it.
I will present the theses of two watershed works presented in readable books, Damasio’s Descartes’ Error and LeDoux’s Emotional Brain. Descartes’ Error presents a very simple thesis: You perform a surgical ablation on a piece of someone’s brain (say, to remove a tumor and tissue around it) with the sole resulting effect of an inability to register emotions, nothing else (the IQ and every other faculty remain the same). What you have done is a controlled experiment to separate someone’s intelligence from his emotions. Now you have a purely rational human being unencumbered with feelings and emotions. Let’s watch: Damasio reported that the purely unemotional man was incapable of making the simplest decision. He could not get out of bed in the morning, and frittered away his days fruitlessly weighing decisions. Shock! This flies in the face of everything one would have expected: One cannot make a decision without emotion. Now, mathematics gives the same answer: If one were to perform an optimizing operation across a large collection of variables, even with a brain as large as ours, it would take a very long time to decide on the simplest of tasks. So we need a shortcut; emotions are there to prevent us from temporizing. Does it remind you of Herbert Simon’s idea? It seems that the emotions are the ones doing the job. Psychologists call them “lubricants of reason.” Joseph LeDoux’s theory about the role of emotions in behavior is even more potent: Emotions affect one’s thinking. He figured out that much of the connections from the emotional systems to the cognitive systems are stronger than connections from the cognitive systems to the emotional systems. The implication is that we feel emotions (limbic brain) then find an explanation (neocortex). As we saw with Claparède’s discovery, much of the opinions and assessments that we have concerning risks may be the simple result of emotions.
People who are as close to being criminal as probability laws can allow us to infer (that is, with a confidence that exceeds the shadow of a doubt) are walking free because of our misunderstanding of basic concepts of the odds. Equally, you could be convicted for a crime you never committed, again owing to a poor reading of probability—for we still cannot have a court of law properly compute the joint probability of events (the probability of two events taking place at the same time). I was in a dealing room with a TV set turned on when I saw one of the lawyers arguing that there were at least four people in Los Angeles capable of carrying O. J. Simpson’s DNA characteristics (thus ignoring the joint set of events—we will see how in the next paragraph). I then switched off the television set in disgust, causing an uproar among the traders. I was under the impression until then that sophistry had been eliminated from legal cases thanks to the high standards of republican Rome. Worse, one Harvard lawyer used the specious argument that only 10% of men who brutalize their wives go on to murder them, which is a probability unconditional on the murder (whether the statement was made out of a warped notion of advocacy, pure malice, or ignorance is immaterial). Isn’t the law devoted to the truth? The correct way to look at it is to determine the percentage of murder cases where women were killed by their husbands and had previously been battered by them (that is, 50%)—for we are dealing with what is called conditional probabilities; the probability that O. J. killed his wife conditional on the information of her having been killed, rather than the unconditional probability of O. J. killing his wife. How can we expect the untrained person to understand randomness when a Harvard professor who deals and teaches the concept of probabilistic evidence can make such an incorrect statement? More particularly, where jurors (and lawyers) tend to make mistakes, along with the rest of us, is in the notion of joint probability. They do not realize that evidence compounds. The probability of my being diagnosed with respiratory tract cancer and being run over by a pink Cadillac in the same year, assuming each one of them is 1/100,000, becomes 1/10,000,000,000—by multiplying the two (obviously independent) events. Arguing that O. J. Simpson had 1/500,000 chance of not being the killer from the blood standpoint (remember the lawyers used the sophistry that there were four people with such blood types walking around Los Angeles) and adding to it the fact that he was the husband of the person and that there was additional evidence, then (owing to the compounding effect) the odds against him rise to several trillion trillion. “Sophisticated” people make worse mistakes. I can surprise people by saying that the probability of the joint event is lower than either. Recall the availability heuristic: with the Linda problem rational and educated people finding the likelihood of…
As an option trader, I have noticed that people tend to undervalue options as they are usually unable to correctly mentally evaluate instruments that deliver an uncertain payoff, even when they are fully conscious of the mathematics. Even regulators reinforce such ignorance by explaining to people that options are a decaying or wasting asset. Options that are out of the money are deemed to decay, by losing their premium between two dates.
Consider that I buy the option for $1. What do I expect the value of the option to be one month from now? Most people think 0. That is not true. The option has a high probability, say 90%, of being worth 0 at expiration, but perhaps 10% probability to be worth an average of $10. Thus, selling the option to me for $1 does not provide the seller with free money. If the seller had instead bought the stock himself at $100 and waited the month, he could have sold it for $120. Making $1 now was hardly, therefore, free money. Likewise, buying it is not a wasting asset. Even professionals can be fooled. How? They confuse the expected value and the most likely scenario (here the expected value is $1 and the most likely scenario is for the option to be worth 0). They mentally overweigh the state that is the most likely, namely, that the market does not move at all. The option is simply the weighted average of the possible states the asset can take.
There is another type of satisfaction provided by the option seller. It is the steady return and the steady feeling of reward—what psychologists call flow. It is very pleasant to go to work in the morning with the expectation of being up some small money. It requires some strength of character to accept the expectation of bleeding a little, losing pennies on a steady basis even if the strategy is bound to be profitable over longer periods. I noticed that very few option traders can maintain what I call a “long volatility” position, namely a position that will most likely lose a small quantity of money at expiration, but is expected to make money in the long run because of occasional spurts. I discovered very few people who accepted losing $1 for most expirations and making $10 once in a while, even if the game were fair (i.e., they made the $10 more than 9.1% of the time). I divide the community of option traders into two categories: premium sellers and premium buyers. Premium sellers (also called option sellers) sell options, and generally make steady money, like John in Chapters 1 and 5. Premium buyers do the reverse. Option sellers, it is said, eat like chickens and go to the bathroom like elephants. Alas, most option traders I encountered in my career are premium sellers—when they blow up it is generally other people’s money.
In addition, there is the risk ignorance factor. Scientists have subjected people to tests—what I mentioned in the prologue as risk taking out of underestimating the risks rather than courage. The subjects were asked to predict a range for security prices in the future, an upper bound and a lower bound, in such a way that they would be comfortable with 98% of the security ending inside such range. Of course violations to such bound were very large, up to 30%. Such violations arise from a far more severe problem: People overvalue their knowledge and underestimate the probability of their being wrong.
A journalist is trained in methods to express himself rather than to plumb the depth of things—the selection process favors the most communicative, not necessarily the most knowledgeable. My medical doctor friends claim that many medical journalists do not understand anything about medicine and biology, often making mistakes of a very basic nature. I cannot confirm such statements, being myself a mere amateur (though at times a voracious reader) in medical research, but I have noticed that they almost always misunderstand the probabilities used in medical research announcements. The most common one concerns the interpretation of evidence. They most commonly get mixed up between absence of evidence and evidence of absence, a similar problem to the one we saw in Chapter 9. How? Say I test some chemotherapy, for instance Fluorouracil, for upper respiratory tract cancer, and find that it is better than a placebo, but only marginally so; that (in addition to other modalities) it improves survival from 21 per 100 to 24 per 100. Given my sample size, I may not be confident that the additional 3% survival points come from the medicine; it could be merely attributable to randomness. I would write a paper outlining my results and saying that there is no evidence of improved survival (as yet) from such medicine, and that further research would be needed. A medical journalist would pick it up and claim that one Professor N. N. Taleb found evidence that Fluorouracil does not help, which is entirely opposite to my intentions. Some naive doctor in Smalltown, even more uncomfortable with probabilities than the most untrained journalist, would pick it up and build a mental block against the medication, even when some researcher finally finds fresh evidence that such medicine confers a clear survival advantage.
This brings to mind another common violation of probability by prime-time TV financial experts, who may be selected for their looks, their charisma, and their presentation skills, but certainly not for their incisive minds. For instance, a fallacy that I saw commonly made by a prominent TV financial guru goes as follows: “The average American is expected to live seventy-three years. Therefore if you are sixty-eight you can expect to live five more years, and should plan accordingly.” She went into precise prescriptions of how the person should invest for a five-more-years horizon. Now what if you are eighty? Is your life expectancy minus seven years? What these journalists confuse is the unconditional and conditional life expectancy. At birth, your unconditional life expectancy may be seventy-three years. But as you advance in age and do not die, your life expectancy increases along with your life. Why? Because other people, by dying, have taken your spot in the statistics, for expectation means average. So if you are seventy-three and are in good health, you may still have, say, nine years in expectation. But the expectation would change, and at eighty-two, you will have another five years, provided of course you are still alive. Even someone one hundred years old still has a positive conditional life expectation. Such a statement, when one thinks about it, is not too different from the one that says: Our operation has a mortality rate of 1%. So far we have operated on ninety-nine patients with great success; you are our one hundreth, hence you have a 100% probability of dying on the table.
I have a trick to know if something real in the world is taking place. I have set up my Bloomberg monitor to display the price and percentage change of all relevant prices in the world: currencies, stocks, interest rates, and commodities. By dint of looking at the same setup for years, as I keep the currencies in the upper left corner and the various stock markets on the right, I managed to build an instinctive way of knowing if something serious is going on. The trick is to look only at the large percentage changes. Unless something moves by more than its usual daily percentage change, the event is deemed to be noise. Percentage moves are the size of the headlines. In addition, the interpretation is not linear; a 2% move is not twice as significant an event as 1%, it is rather like four to ten times. A 7% move can be several billion times more relevant than a 1% move! The headline of the Dow moving by 1.3 points on my screen today has less than one billionth of the significance of the serious 7% drop of October 1997. People might ask me: Why do I want everybody to learn some statistics? The answer is that too many people read explanations. We cannot instinctively understand the nonlinear aspect of probability.
What is the mechanism that should convince authors to avoid reading comments on their work, except for those they solicit from specified persons for whom they have intellectual respect? The mechanism is a probabilistic method called conditional information: Unless the source of the statement has extremely high qualifications, the statement will be more revealing of the author than the information intended by him. This applies, of course, to matters of judgment. A book review, good or bad, can be far more descriptive of the reviewer than informational about the book itself. This mechanism I also call Wittgenstein’s ruler: Unless you have confidence in the ruler’s reliability, if you use a ruler to measure a table you may also be using the table to measure the ruler. The less you trust the ruler‘s reliability (in probability called the prior),the more information you are getting about the ruler and the less about the table. The point extends way beyond information and probability. This conditionality of information is central in epistemology, probability, even in studies of consciousness. We will see later extensions with “ten sigma” problems.
Recall that the accomplishment from which I derive the most pride is my weaning myself from television and the news media. I am currently so weaned that it actually costs me more energy to watch television than to perform any other activity, like, say, writing this book. But this did not come without tricks. Without tricks I would not escape the toxicity of the information age. In the trading room of my company, I have the television set turned on all day with the financial news channel CNBC staging commentator after commentator and CEO after CEO murdering rigor all day long. What is the trick? I have the volume turned completely off. Why? Because when the television set is silent, the babbling person looks ridiculous, exactly the opposite effect as when the sound is on. One sees a person with moving lips and contortions in his facial muscles, taking themselves seriously—but no sound comes out. We are visually but not auditorily intimidated, which causes a dissonance. The speaker’s face expresses some excitement, but since no sound comes out, the exact opposite is conveyed. This is the sort of contrast the philosopher Henri Bergson had in mind in his Treatise on Laughter, with his famous description of the gap between the seriousness of a gentleman about to walk on a banana skin and the comical aspect of the situation. Television pundits lose their intimidating effect;they even look ridiculous. They seem to be excited about something terribly unimportant. Suddenly pundits become clowns, which is a reason the writer Graham Greene refused to go on television.
Say you own a painting you bought for $20,000, and owing to rosy conditions in the art market, it is now worth $40,000. If you owned no painting, would you still acquire it at the current price? If you would not, then you are said to be married to your position. There is no rational reason to keep a painting you would not buy at its current market rate—only an emotional investment. Many people get married to their ideas all the way to the grave. Beliefs are said to be path dependent if the sequence of ideas is such that the first one dominates.
Again, compare them with Soros, who walks around telling whoever has the patience to listen to him that he is fallible. My lesson from Soros is to start every meeting at my boutique by convincing everyone that we are a bunch of idiots who know nothing and are mistake-prone, but happen to be endowed with the rare privilege of knowing it.
A more human version can be read in Seneca’s Letters from a Stoic, a soothing and surprisingly readable book that I distribute to my trader friends (Seneca also took his own life when cornered by destiny).
There seems to be some evidence that conversations and correspondence with intelligent people is a better engine for personal edification than plain library-ratting (human warmth: Something in our nature may help us grow ideas while dealing and socializing with other people).
To view it in another way, consider the difference between judging on process and judging on results. Lower-ranking persons in the enterprise are judged on both process and results—in fact, owing to the repetitive aspect of their efforts, their process converges rapidly to results. But top management is only paid on result—no matter the process. There seems to be no such thing as a foolish decision if it results in profits. “Money talks,” we are often told. The rest is supposed to be philosophy.
It took me an entire lifetime to find out what my generator is. It is: We favor the visible, the embedded, the personal, the narrated, and the tangible; we scorn the abstract. Everything good (aesthetics, ethics) and wrong (Fooled by Randomness) with us seems to flow from it.