Our quote of the day comes from an article in this Sunday’s NYT magazine, Don’t Blink! The Hazards of Confidence by Daniel Kahneman:

“The illusion of skill is not only an individual aberration; it is deeply ingrained in the culture of the [financial] industry. Facts that challenge such basic assumptions — and thereby threaten people’s livelihood and self-esteem — are simply not absorbed. The mind does not digest them. This is particularly true of statistical studies of performance, which provide general facts that people will ignore if they conflict with their personal experience.”

I find that, unfortunately, to be terribly true.

For those of you who may be unfamiliar with Kahneman, he is a professor at Princeton and Nobel laureate. He is notable for his work on the psychology of judgment and decision-making, and behavioral economics.

Category: Finance, Philosophy, Psychology

Please use the comments to demonstrate your own ignorance, unfamiliarity with empirical data and lack of respect for scientific knowledge. Be sure to create straw men and argue against things I have neither said nor implied. If you could repeat previously discredited memes or steer the conversation into irrelevant, off topic discussions, it would be appreciated. Lastly, kindly forgo all civility in your discourse . . . you are, after all, anonymous.

26 Responses to “QOTD: The Illusion of Skill”

  1. Jack Damn says:

    “It’s human nature to find patterns where there are none and to find skill where luck is a more likely explanation.” ~William Bernstein

  2. BusSchDean says:

    Kahneman (with Tversky) did a little experiment that showed people would drive across town to save $5 on a $15 item but would not make the same trip to save $5 on a $125 item. Economics couldn’t explain it because for them it was “drive across town = $5 savings” in each case. He does wonderful work.

  3. ellidc says:

    In the article he says no one at the firm seemed to be aware of the nature of the game that its stock pickers were playing. But I wonder if in some ways Kahneman is closing his eyes to the real game. He is looking at investment returns as though it was the only game being played, but the wider game for the management of the firm is increasing assets under management and maximizing the the amount extracted for themselves. In that case the illusion of skill is not always a problem of self delusion or cognitive quirk, but a marketing and wealth maximizing strategy.

  4. mikerhak says:

    Last paragraph – “Is the environment in which the judgment is made sufficiently regular to enable predictions from the available evidence?”

    This makes a strong argument that poker should be legalized in every state and that it is the stock trading which should be outlawed.

    I develop software for my employer who is in the business of analyzing investment manager returns. Yes, basically automating the analysis of coin flips. But it makes fancy reports to distribute to clients.

  5. Transor Z says:

    Maybe it was just the editor’s choice of story title, referencing Malcolm Gladwell, but there’s a fallacy of false alternatives being presented here. The issue is finding the right decisionmaking tool for the job, not picking up a hammer to the exclusion of the screwdriver and treating everything like a nail.

    The army screening story is a nice example of junk science. The wrong metrics being collected based on a flawed hypothesis with highly subjective data collection and with feedback methodology that clearly lacked a QA process component to remedy a bad system.

    There are times when Gladwell’s “Blink” processes are highly useful because they are an amazingly efficient and accurate use of limited data. But I don’t think anyone would ever seriously argue that “blink” is a universal decisionmaking tool. Clearly, there are lots of examples of when it sucks.

  6. Clem Stone says:

    I’m just hoping that the illusion also applies to LACK of skill.

  7. jmacdon says:

    The analysis below is ridiculous:

    “To find the answer, I computed the correlations between the rankings of advisers in different years, comparing Year 1 with Year 2, Year 1 with Year 3 and so on up through Year 7 with Year 8. That yielded 28 correlations, one for each pair of years. While I was prepared to find little year-to-year consistency, I was still surprised to find that the average of the 28 correlations was .01. In other words, zero. The stability that would indicate differences in skill was not to be found. The results resembled what you would expect from a dice-rolling contest, not a game of skill. ”

    As an extreme example, let’s surmise that the advisors being compared were the top 25 advisors in the whole world, beating all others handily, year after year. However, they were pretty well matched within that group of 25, so a given advisor might be ranked first one year and fifth the next. In this example the 25 advisors are still the 25 best advisors in the world, they are just inconsistent at besting their co-workers.

  8. darth beta says:

    Like most studies it lacks any context. Humans are highly complex and use decades of accumulated knowledge to make decision. At the least we are pragmatic algorithmic decision makers. And algorithms are defined by their inputs. In humans the input is experience. The human decision making process is simply to complex to study.

    There is this fallacy that science can understand human decision making. It can’t. (Ask the guys who had studies that said the Iraq war would be over in a short time horizon)

    There is no way to measure why some body did something only to measure the decision. Lets use the $5 discount example. Yes it is $5 either way but what are the externalities and experience that go into that decision making process?

    Maybe the $125 dollar item was heavy and one store offer help lifting into the car.
    Maybe there is a cute girl at one store.
    Maybe some sort of allegiance.
    Maybe the store has a better return policy ect.

    And it goes on and on.

    Gladwell and others make for interesting conversation but they lack any context or empathy.

    Forget the studies, get the guy on the ground recommendation and build the context.

  9. dead hobo says:

    Addendum

    Experts always come from somewhere else

    The people you feel the most warmly towards are the most right

    People you don’t like are usually full of crap

    Most people who challenge you need to be kicked in the nuts

    Providence will always provide

  10. BusSchDean says:

    darth: All of which is easily controlled for in an experimental design. That’s what experimental designs do…in testing drugs and is some (not all) behavioral tests. They isolate one manipulation while controlling for the rest.

  11. darth beta says:

    BusSch- exactly my point science needs isolation and life is precisely not that!

  12. BusSchDean says:

    Ergo all science is pointless? Congratulations! There is a political party for you.

    Naturally isolating an effect does not explain complex behavior but once the effect is isolated and replicated (i.e., can be repead time and again) it because a known variable to look for — one that was not known before. Over time we can actually begin to understand behavior. I would be the first to say there is much we do not know but Kahneman has helped economics move away from model based on false assumptions about human behavior. Knowledge often builds by inches over time.

  13. BusSchDean says:

    ..in becomes….

  14. BusSchDean says:

    Gee…I do apologize for all of the typos…its Friday!

  15. Jim Greeen says:

    From William Goldman…”No one knows”

  16. 873450 says:

    “Derivatives and the related application of more-sophisticated approaches to measuring and managing risk are key factors underpinning the greater resilience of our largest financial institutions. Derivatives permit the unbundling of financial risks.” – Alan Greenspan (2005)

    Don’t worry. Be happy. We know what we’re doing.

  17. BusSchDean says:

    873…. :-) Of course “we” do. Its just that the definition of “we” matters a great deal. Stiglitz, Schiller, Rubini all have been right more than once regarding the macro economy both before and after the bubble, with explanations that hold up.

    “We” have two problems that “we” seem not to want to admit. First, power matters and who we give power to matters. Once Greenspan was given power by Reagan (not one known to worry too much about the facts) many people were loathe to question him. Only now everyone seems to know he wasn’t up to the job either from a theoretical or practical perspective. Second, and I just had to deal with this today, we rely more on labels then ever before. Depending on our camp, the mere fact that someone is labeled to be in our camp, or not, gives instant credibility. No one really looks at the quality of the work. Is this because we are lazy? scared? tribalistic?

  18. ToNYC says:

    “Forget the studies, get the guy on the ground recommendation and build the context.”

    The only way to get first-hand information is by showing up. If you build or share your dream on less than that quality of information, you incorporate the fail.

  19. BusSchDean says:

    “Forget the studies, get the guy on the ground recommendation…” Indeed. The problem with this is that the people who give this advice have already gotten and digested the studies, which is why they are ahead of the guy on the ground. Here is a simple question (maybe a bit feisty tonight): How do you define “first-hand information”?

  20. ToNYC says:

    from Websters Encyclopedic Unabridged ..”first-hand” being “direct from the original source.” An apple from the tree (of knowledge). Unprocessed. As I remember from the Street, “Before opening mouth, engage brain.”

  21. BusSchDean says:

    So then presumably Rajaratnam “earned” his way if he got his information direct from the original source? It appears that all he needed to do was know the right person. How many of the predictive models used in finance were first tested extensively with simulated “contexts”? And just how did those HFT guys who now generate more than 50% of the volume build their models (not that I approve)? Remember my argument was not that we only learn from studies, mine was that Kahneman’s studies help us better understand human behavior.

    We learn in many ways, always have. Galileo did tons of tightly constructed studies is that somehow less legitimate than watching an apple fall? Much can be learned by being in the context no doubt but that in now way means studies are not, can not or have not been used successfully in finance and just about everywhere else. A HUGE number of practical innovations came from what first started in the lab as “studies” in support of NASA. Studies that you disparage have done everything from give you the iPad you enjoy to the cholesterol medicine to how people shop online to how an airplane pilot will tend to react to a specific situation. You might like the movie Longitude as it documents how a culture that ignores studies would for years send men to sea with little hope of knowing where they were when one man with a fist full of data he generated on land could have changed that.

  22. ToNYC says:

    All it takes is one person with ethics and the morality learned by age seven in a group do stay on course respecting the rights of others.

  23. BusSchDean says:

    Really? Write that after reading “Everyman Dies Alone,” a novel based on the true story of a husband and wife in Nazi Berlin who initiated an anti-Nazi postcard writing campaign. Just two people, ethical and moral, doing what they could against a society mostly willing to facilitate or at least tolerate extreme fascism. I will spoil the ending for you…after hundreds of postcards they are caught by the Gestapo and killed. It clearly takes more than one person.

  24. ToNYC says:

    You and me makes two.

  25. BusSchDean says:

    We’re done.

  26. ToNYC says:

    Do I get graded?
    I earned my M.B.E. at Myown U. with all the rights and privileges hereto pertaining.