maxjmartin.com

Fooled by Randomness

Book notes for “Fooled by Randomness”, The Hidden Role of Chance in Life and in the Markets

Quick review:

While many of the biases discussed were familiar from other works (Kahneman crops up everywhere), the focus on randomness and black-swan events was interesting and useful. I feel like I have internalised the lesson. although in practice, maybe not.

The writing was fun, although I can see why some people might not like it.

Highlights:

It is as if there were two planets: the one in which we actually live and the one, considerably more deterministic, on which people are convinced we live. location 265

Trading forces someone to think hard; those who merely work hard generally lose their focus and intellectual energy. In addition, they end up drowning in randomness; work ethics, Nero believes, draw people to focus on noise rather than the signal location 683

One can illustrate the strange concept of alternative histories as follows. Imagine an eccentric (and bored) tycoon offering you $10 million to play Russian roulette, i.e., to put a revolver containing one bullet in the six available chambers to your head and pull the trigger. Each realization would count as one history, for a total of six possible histories of equal probabilities. Five out of these six histories would lead to enrichment; one would lead to a statistic, that is, an obituary with an embarrassing (but certainly original) cause of death. The problem is that only one of the histories is observed in reality; and the winner of $10 million would elicit the admiration and praise of some fatuous journalist (the very same ones who unconditionally admire the Forbes 500 billionaires). location 843

$10 million earned through Russian roulette does not have the same value as $10 million earned through the diligent and artful practice of dentistry. They are the same, can buy the same goods, except that one’s dependence on randomness is greater than the other. To an accountant, though, they would be identical; to your next-door neighbor too. Yet, deep down, I cannot help but consider them as qualitatively different. location 861

In a now famous experiment they found that the majority of people, whether predictors or nonpredictors, will judge a deadly flood (causing thousands of deaths) caused by a California earthquake to be more likely than a fatal flood (causing thousands of deaths) occurring somewhere in North America (which happens to include California). As a derivatives trader I noticed that people do not like to insure against something abstract; the risk that merits their attention is always something vivid. location 1055

Borrowed wisdom can be vicious. I need to make a huge effort not to be swayed by well-sounding remarks. location 1086

Mathematics is principally a tool to meditate, rather than to compute. location 1145

If you think that merely reading history books would help you learn “from other’s mistakes,” consider the following nineteenth-century experiment. In a well-known psychology case the Swiss doctor Claparède had an amnesic patient completely crippled with her ailment. Her condition was so bad that he would have to reintroduce himself to her at a frequency of once per fifteen minutes for her to remember who he was. One day he secreted a pin in his hand before shaking hers. The next day she quickly withdrew her hand as he tried to greet her, but still did not recognize him. Since then plenty of discussions of amnesic patients show some form of learning on the part of people without their being aware of it and without it being stored in conscious memory. The scientific name of the distinction between the two memories, the conscious and the nonconscious, is declarative and nondeclarative. Much of the risk avoidance that comes from experiences is part of the second. The only way I developed a respect for history is by making myself aware of the fact that I was not programmed to learn from it in a textbook format. location 1268

There is nothing wrong with a risk taker taking a hit provided one declares that one is a risk taker rather than that the risk being taken is small or nonexistent. location 1284

Now the civil servant called the trades that ended up as losers “gross mistakes,” just like journalists call decisions that end up costing a candidate his election a “mistake.” I will repeat this point until I get hoarse: A mistake is not something to be determined after the fact, but in the light of the information until that point. location 1317

Bad trades catch up with you, it is frequently said in the markets. Mathematicians of probability give that a fancy name: ergodicity. It means, roughly, that (under certain conditions) very long sample paths would end up resembling each other. The properties of a very, very long sample path would be similar to the Monte Carlo properties of an average of shorter ones. location 1337

If there is anything better than noise in the mass of “urgent” news pounding us, it would be like a needle in a haystack. People do not realize that the media is paid to get your attention. For a journalist, silence rarely surpasses any word. On the rare occasions when I boarded the 6:42 train to New York I observed with amazement the hordes of depressed business commuters (who seemed to prefer to be elsewhere] studiously buried in The Wall Street Journal, apprised of the minutiae of companies that, at the time of writing now, are probably out of business. Indeed it is difficult to ascertain whether they seem depressed because they are reading the newspaper, or if depressive people tend to read the newspaper, or if people who are living outside their genetic habitat both read the newspaper and look sleepy and depressed. But while early on in my career such focus on noise would have offended me intellectually, as I would have deemed such information as too statistically insignificant for the derivation of any meaningful conclusion, I currently look at it with delight. I am happy to see such mass-scale idiotic decision making, prone to overreaction in their postperusal investment orders location 1373

A 15% return with a 10% volatility (or uncertainty) per annum translates into a 93% probability of success in any given year. But seen at a narrow time scale, this translates into a mere 50.02% probability of success over any given second as shown in Table 3.1. Over the very narrow time increment, the observation will reveal close to nothing. Yet the dentist’s heart will not tell him that. location 1458

Over a short time increment, one observes the variability of the portfolio, not the returns. In other words, one sees the variance, little else. location 1476

The same methodology can explain why the news (the high scale) is full of noise and why history (the low scale) is largely stripped of it (though fraught with interpretation problems). location 1485

This was one of the small points that emerging-market economists around the globe, from talking to each other so much, forgot to take into account. Veteran trader Marty O’Connell calls this the firehouse effect. He had observed that firemen with much downtime who talk to each other for too long come to agree on many things that an outside, impartial observer would find ludicrous (they develop political ideas that are very similar). Psychologists give it a fancier name, but my friend Marty has no training in behavioral sciences. location 1712

An overestimation of the accuracy of their beliefs in some measure, either economic (Carlos) or statistical (John). They never considered that the fact that trading on economic variables has worked in the past may have been merely coincidental, or, perhaps even worse, that economic analysis was fit to past events to mask the random element in it. location 1800

A tendency to get married to positions. There is a saying that bad traders divorce their spouse sooner than abandon their positions. Loyalty to ideas is not a good thing for traders, scientists–or anyone. location 1813

No precise game plan ahead of time as to what to do in the event of losses. They simply were not aware of such a possibility. Both bought more bonds after the market declined sharply, but not in response to a predetermined plan. location 1820

I resent the person who, without having done much homework in libraries, thinks that he is onto something rather original and insightful on a given subject matter (and I respect people with scientific minds, like my friend Stan Jonas, who feel compelled to spend their nights reading wholesale on a subject matter, trying to figure out what was done on the subject by others before emitting an opinion—would the reader listen to the opinion of a doctor who does not read medical papers?). location 1923

Let us assume that the reader shared my opinion, that the market over the next week had a 70% probability of going up and 30% probability of going down. However, let us say that it would go up by 1% on average, while it could go down by an average of 10%. What would the reader do? Is the reader bullish or bearish? location 1939

Accordingly, bullish or bearish are terms used by people who do not engage in practicing uncertainty, like the television commentators, or those who have no experience in handling risk. Alas, investors and businesses are not paid in probabilities; they are paid in dollars. location 1943

The best description of my lifelong business in the market is “skewed bets,” that is, I try to benefit from rare events, events that do not tend to repeat themselves frequently, but, accordingly, present a large payoff when they occur. I try to make money infrequently, as infrequently as possible, simply because I believe that rare events are not fairly valued, and that the rarer the event, the more undervalued it will be in price. In addition to my own empiricism, I think that the counterintuitive aspect of the trade (and the fact that our emotional wiring does not accommodate it) gives me some form of advantage. Why are these events poorly valued? Because of a psychological bias; people who surrounded me in my career were too focused on memorizing section 2 of The Wall Street Journal during their train ride to reflect properly on the attributes of random events. location 1967

On the surface, my statement here may seem to contradict earlier discussions, where I blame people for not learning enough from history. The problem is that we read too much into shallow recent history, with statements like “this has never happened before,” but not from history in general (things that never happened before in one area tend eventually to happen). In other words, history teaches us that things that never happened before do happen. location 2035

Psychologists recently found out that people tend to be sensitive to the presence or absence of a given stimulus rather than its magnitude. This implies that a loss is first perceived as just a loss, with further implications later. The same with profits. The agent would prefer the number of losses to be low and the number of gains to be high, rather than optimizing the total performance. location 2076

You can more safely use the data to reject than to confirm hypotheses. Why? Consider the following statements: Statement A: No swan is black, because I looked at four thousand swans and found none. Statement B: Not all swans are white. I cannot logically make statement A, no matter how many successive white swans I may have observed in my life and may observe in the future (except, of course, if I am given the privilege of observing with certainty all available swans). It is, however, possible to make Statement B merely by finding one single counterexample. location 2184

So why do we consider the worst case that took place in our own past as the worst possible case? If the past, by bringing surprises, did not resemble the past previous to it (what I call the past’s past), then why should our future resemble our current past? location 2203

I suddenly felt financially insecure and feared becoming an employee of some firm that would turn me into a corporate slave with “work ethics” (whenever I hear work ethics I interpret inefficient mediocrity). location 2261

But, as usual, beware the middlebrow: A small knowledge of probability can lead to worse results than no knowledge at all. location 2383

Recall that the survivorship bias depends on the size of the initial population. The information that a person derived some profits in the past, just by itself, is neither meaningful nor relevant. We need to know the size of the population from which he came. In other words, without knowing how many managers out there have tried and failed, we will not be able to assess the validity of the track record. If the initial population includes ten managers, then I would give the performer half my savings without a blink. If the initial population is composed of 10,000 managers, I would ignore the results. location 2649

The most intuitive way to describe the data mining problem to a nonstatistician is through what is called the birthday paradox, though it is not really a paradox, simply a perceptional oddity. If you meet someone randomly, there is a one in 365.25 chance of your sharing their birthday, and a considerably smaller one of having the exact birthday of the same year. So, sharing the same birthday would be a coincidental event that you would discuss at the dinner table. Now let us look at a situation where there are 23 people in a room. What is the chance of there being 2 people with the same birthday? About 50%. For we are not specifying which people need to share a birthday; any pair works. location 2682

I am currently sitting on a beach in Copacabana, in Rio de Janeiro, attempting to do nothing strenuous, away from anything to read and write (unsuccessfully, of course, as I am mentally writing these lines). location 2871

love the phrasing

Consider that those who started theorizing upon seeing a tiger on whether the tiger was of this or that taxonomic variety, and the degree of danger it represented, ended up being eaten by it. Others who just ran away at the smallest presumption and were not slowed down by the smallest amount of thinking ended up either outchasing the tiger or outchasing their cousin who ended up being eaten by it. location 3054

example also appears in AI-Z, wonder if it is from a common source?

Proust wrote frequently about the surprise people have when coming across emotions in Homeric heroes that are similar to those we experience today. By genetic standards, these Homeric heroes of thirty centuries ago in all likelihood have the exact identical makeup as the pudgy middle-aged man you see schlepping groceries in the parking lot. More than that. In fact, we are truly identical to the man who perhaps eighty centuries ago started being called “civilized,” in that strip of land stretching from southeastern Syria to southwestern Mesopotamia. location 3244

He figured out that much of the connections from the emotional systems to the cognitive systems are stronger than connections from the cognitive systems to the emotional systems. The implication is that we feel emotions (limbic brain) then find an explanation (neocortex). location 3310

The epiphany I had in my career in randomness came when I understood that I was not intelligent enough, nor strong enough, to even try to fight my emotions. Besides, I believe that I need my emotions to formulate my ideas and get the energy to execute them. I am just intelligent enough to understand that I have a predisposition to be fooled by randomness—and to accept the fact that I am rather emotional. location 3534

One of the most irritating conversations I’ve had is with people who lecture me on how I should behave. Most of us know pretty much how we should behave. It is the execution that is the problem, not the absence of knowledge. I am tired of the moralizing slow-thinkers who pound me with platitudes like I should floss daily, eat my regular apple, and visit the gym outside of the New Year’s resolution. In the markets the recommendation would be to ignore the noise component in the performance. We need tricks to  get us there but before that we need to accept the fact that we are mere animals in need of lower forms of tricks, not lectures. location 3684

yes!

People confuse science and scientists. Science is great, but individual scientists are dangerous. They are human; they are marred by the biases humans have. Perhaps even more. location 3838

Assume that two equally charismatic, empty-suit-style twin brothers manage to climb the corporate ladder to get two different jobs in two different corporations. Assume that they own good-looking suits, that they have MBAs, and that they are tall (the only truly visible predictor of corporate success is to be taller than average). location 3962

I confess that, as a practitioner of randomness, I focused primarily on the defects of my own thinking (and that of a few people I’ve observed or tracked through time). I also intended the book to be playful, which is not very compatible with referencing every idea to some scientific paper to give it a degree of respectability. location 4064