“He can take a model and turn it into a narrative”

 

Right after the election, Felix wrote a post “When quants tell stories.” Clever as it was, I had issues with the underlying premise – namely, that the value of Nate Silver’s modeling lay more in the narrative tale as told by Silver himself and not the underlying  mathematics of statistical probability analysis.

As I read this, I kept thinking “Not exactly.”

I have been meaning to address this for some time, but a hurricane and a few weeks of no electricity got in the way. Before this gets away from me, I wanted to look back to see where I concur with Felix’ assessment, and where we disagree. It is an interesting story and an important idea – but there are a few specific points that are worth discussing.

Let’s start with the money quote:

“If you think that the value of Nate Silver is in the model, you’re missing the most important part: there are lots of people with models, and most of those models are pretty similar to each other. The thing which sets Silver apart from the rest is that he can write: he can take a model and turn it into a narrative, walking his readers through to his conclusions.”

That’s the paragraph that bothered me the most. We both agree that Nate can write – he is an effective communicator, informing his readers about a moderately complex statistical model in a way that is both informative and entertaining.

Its the rest of that paragraph that I had an issue with:

-Lots of people had models, which ranged from okay to terrible
-Most of the models differed significantly from each other
-Lots of people told interesting stories, but they got the narrative and the conclusions wrong
-A few people had models that got it right; some were good, others lucky.

Despite those errors, what caught my interest as an investor was the discussion about Narratives. It is in exactly this context that I want to discuss what a “Narrative” is conceptually, and what it means to those who toil in the Capital Markets.

It is more than just a story, rather it was a way to look at and describe the world:

-Narratives are about selling a perspective
-Narratives typically focus on squishier, less quantifiable aspects of an issue
-Narratives often hit emotional buttons, making the reader feel good about the story.
-Narratives are about the outcome, not the process.
-Models are about process;
-Good models produce a consistent process, which is more important than any single result.

In the world of investing, the Narrative is how a given stock or investment thesis gets sold. Narratives are how the threads of the story weave a tapestry that leads to an action: Buy this stock, sell this mutual fund, vote for this candidate.

Whenever a broker is pitching a stock to someone, there will invariably be a statement along the lines of “What’s the story with this stock?”

The answer usually involves a narrative that discusses some combination of the following:

-Management Team
-New Products
-Growth over competitors
-Strategic Acquisitions
-Penetrating new markets
-Value proposition
-Earnings Story
-Expansion overseas (China!)
-M&A target

Taken collectively, the stock narrative makes for a compelling story. Where it falls apart is when any of these items are closely scrutinized — the past track record of these inputs is found wanting.

Ask yourself: Who has the acumen to look at a management team and draw a conclusion about future stock growth? Almost nobody (in my experience). What is the narrator’s track record doing stock picking based on just that (or anyone elses for that matter)? The Narrator’s assessment of new products, markets, takeovers, etc. — how good has it been? These hot button issues simply fall apart as a method of selecting stocks that will outperform over either the short or long term.

However, there is an enormous difference between being able to communicate what the mathematics is informing you of, and living and dying by a narrative (often sans data). Consider Peggy Noonan — a compelling wordsmith, best selling author, and former speechwriter for Ronald Reagan. She and other non modeling story tellers constructed terrific narratives about a variety of emotional issues, none of which statistically mattered.

My favorite example was the yard signs in Florida reflecting an advantage for Romney in enthusiasm amongst his political base. There are very obvious modeling issues with that: What was the method for measuring those signs? How widespread was this nationally, and more importantly, in swing states? What is the past correlation between numerical signage advantages and election outcomes, etc?

Looked at that way, the narrative approach which only feigned a measure of statistical accuracy utterly failed. The tellers of these tales helped make themselves feel better emotionally, but these narrations did nothing for their forecasting acumen.

And that is the primary problems with Narratives: They exist to make us feel better — about a stock purchase, an election or even ourselves.

Humans are fond of using narratives to describe the world around them. Perhaps it is a vestige left over from when we had no written language. Narratives were how we passed along crucial information in a memorable way. The depiction of facts and figures simply was not a compelling form of communication to hunter gatherers.

Investors need to remember this whenever they are told a story. Narrative may be entertaining, but they won’t make you money.

 

 
Previously:
Lessons from the 2012 election (Washington Post, November 10 2012)

Nate Silver and the Lessons of 2012 (November 6, 2012)

Source:
When quants tell stories
Felix Salmon
Reuters November 7, 2012
http://blogs.reuters.com/felix-salmon/2012/11/07/when-quants-tell-stories/

Category: Investing, Mathematics, Philosophy, Politics, Quantitative, Really, really bad calls

Please use the comments to demonstrate your own ignorance, unfamiliarity with empirical data and lack of respect for scientific knowledge. Be sure to create straw men and argue against things I have neither said nor implied. If you could repeat previously discredited memes or steer the conversation into irrelevant, off topic discussions, it would be appreciated. Lastly, kindly forgo all civility in your discourse . . . you are, after all, anonymous.

20 Responses to “The Dangers of Non-Modeled Narrative Story Tellers”

  1. Mike in Nola says:

    Correctomundo as usual.
    Glamour seems to play a huge part in stock prices while earnings and management may or may not. MSFT was a perfect example. Glamour stock of the tech bubble but never recovered its allure after that crash despite really good business growth. Earnings tripled between 2001 and 2008 before AAPL became a player. Similarly for CSCO.
    Yet the stocks never really went anywhere.

  2. falconsgyre says:

    One of your better notes. Too bad it didn’t come out closer to the election result when minds were focused on the election narrative instead of the fiscal cliff narrative, though you are dealing well with that as well…

  3. pmorrisonfl says:

    I think the thing is that most people, most of the time, think in terms of narratives (or, more technically, Kahneman’s ‘System 1′ from ‘Thinking, Fast and Slow’). It takes explicit effort to switch over to models/’System 2′… and so the model builders of all types – mathematicians, statisticians, software developers – have trouble conveying the results of the time they spend in System 2 to people who primarily run in System 1 (and even the model builders spend most of their time in System 1.) And so the people with convincing narratives have an edge in persuasion, whether or not it corresponds to a model. I think Salmon is, rightly, trying to say that it is worth treasuring those who both have an accurate model and a convincing story.

  4. GeorgeBurnsWasRight says:

    To me, the issue of Nate Silver’s outperformance is the same as with a fund manager’s outperformance: how much of it is due to superior performance and how much of it is luck.

    By definition in any human endeavor some perform better, some worse, and many are around average. It’s yet to be proven than beyond limited statistical projections that humans can consistently predict the future well enough to be of much value.

    Interestingly, while it’s difficult to prove which activities produce success in making predictions, it is somewhat easier to prove which activities produce failure. Barry, I believe your greatest contribution through your blog has been to identify some of these failure-enhancing activities. Logically (a slippery word!), avoiding these activities should increase our odds of achieving some success.

  5. gman says:

    Most valuable post Barry has made. Most financial media are pushing a narrative..somebodies narrative that benefits somebodies book.

    Liquidity trap models explained why we would NOT have “hyperinflation and a 10yr crash” but the narratives contra were aired constantly in the usual places! (CNBC, WSJ, IBD)

  6. lunartop says:

    Horses for courses. A narrative may be helpful in recognising the short comings of a model and vica versa. Sometimes a model will work well because the data is suspect, ditto a narrative. Sometimes the model or narrative are reflexive. The problem is recognising when your using a suspect narrative or model to support a dogma – this is called wishful thinking.

  7. capitalistic says:

    Nice point of view. As my financial mathematics professor said, “Every quantitative model is wrong.” So what makes a “good” model? Quality data, low noise and INTERPRETATION. Everything else is a Hail Mary.
    Nate Silver is brilliant: He used good data, eliminated enough noise, and interpreted the results in a cohesive and coherent manner. Simplicity.

  8. capitalistic:

    Remember George Box’s corollary: “All models are wrong, but some are useful.”

  9. Bokolis says:

    A little narrative about the yard signs as it may apply to a swing state…

    In Pennsylvania, or at least the parts I frequent, you typically see yard signs for every office under the sun (and then some); my personal favorite is Recorder of Deeds. Hey, you can’t just leave that up to some ADDled type.

    While it was always apparent that Obama was going to carry the state (and the places/counties I frequent), only Romney signs were on the yards. My cynical ass immediately figured that the white folks- apparently not realizing that a signless yard is just as conspicuous- didn’t want their neighbors knowing that they were voting for the black guy.

    The more virtuous can make the argument that, like a lot of us, despite holding varying opinions as to how ineffective he has been in his first term, those folks were resigned to vote for Obama and their support was lukewarm, certainly not solid enough to put a sign out on the lawn.

    I’ve been there long enough to have shed any sense of virtue.

  10. RW says:

    Stories can get you into big gains or big troubles or nothing at all and oddly enough it doesn’t seem to be related to how good a story it is but how well it translates the world.

    George Steiner’s seminal work on translation and the links between the evolution of language, culture and humanity, makes the case that language did not evolve to communicate factual information — humans derived that purpose secondarily — it evolved to strengthen identity, a bond to the group, and to sustain the group through time, largely with myth.

    That is, one of language’s primary purposes is fictive. This is why it so easy to lie but it is also a key avenue to imagination, the ability to say something as if it were so (even if it is not), and what I think is humanity’s greatest invention: The future tense, the ability to say something as if it will be so.

    This is the key to goal setting: By extension, this will be so if X happens or if I do X.

    Of course this is also how to immeasurably bullshit yourself too: Double-edged sword and all that.

    A scientific attitude (respect for evidence and causal warrant for claims) and a probabilistic habit of thinking helps avoid bullshitting yourself but even that isn’t full proof; language makes it all too easy to lie to your own self too (lord knows I’ve got all the evidence for that I could ever want but probably about as much as I deserve).

  11. BTW, I’ve referenced this repeatedly in the “Monkeys love a Narrative” comments over the years, as well as “Your Brain On Stocks” presentation I have been giving the past 18 months

  12. rd says:

    As an engineer, my basic rule on narratives is that a good narrative can help people accept reality but it won’t save them from reality.

    People are often led down the wrong road by somebody spinning a selling narrative while the data is sitting there screaming that they are stepping into a whirlpool. Engineers, quants, etc. need to learn how to do good narratives otherwise the good models and data will simply end up in filing cabinets while the BS is acted upon because of some slick salesmanship. In the end, only the BS’ers win in that scenario.

    One of the primary reasons the Challenger blew up was that the engineers on the program struggled to come up with a with a winning narrative and a good presentation of their data on why the shuttle should NOT fly given the enormous pressure to launch with the State of the Union speech coming up.

  13. [...] Barry Ritholtz, “Investors need to remember this whenever they are told a story. Narrative may be entertaining, but they won’t make you money.”  (Big Picture) [...]

  14. faulkner says:

    The title and concepts contained within Donald Spence’s 1984 book “Narrative Truth and Historical Truth” are an early summary of your points (and later developments in narrative studies). Effective (ie. compelling) narratives have conventions that take them far from historical accuracy. Descriptions of ‘legendary’ money managers are still more hagiography about them than not. Felix Salmon rightly notes Nate Silver’s literary gifts, and you the distance between models and stories.

    Here’s the thing; models are synchronic – the analysis of structure of a system state. Narrative/stories are diachronic – the analysis of the system changes over time. The model, based on metaphor, is more basic. But the stories that use them have their conventions that we find (almost) irresistible.

  15. pmorrisonfl says:

    > rd: One of the primary reasons the Challenger blew up…

    Very relevant and pertinent example. Feynman’s report (http://www.ralentz.com/old/space/feynman-report.html), an appendix to the Rogers Commission report on Challenger points this out in detail, and ends with the great quote:

    ” For a successful technology, reality must take precedence over public relations, for nature cannot be fooled.”

  16. slowkarma says:

    I think you’re absolutely correct in your general argument.

    Specifically, though…Nate Silver can explain statistical arguments, but his book was full of simple grammatical and structural (writing) errors. He wrote like you’d expect a New York bookie to write…that is, like a high school dropout who is good with numbers.

    Has anybody noticed that Silver *can’t* lose? He had Obama the winner over Romney by something lie 80-20, or four chances if five. If Romney had won, his response would have been, “This was the one time in five.” In over words, he is working just like a bookie — he’ll take either side of the bet: you can put up twenty bucks to win eighty (picking Romney) or eighty bucks to win twenty (picking Obama.)

  17. slowkarma says:

    And here’s a Barry story we really need — they way Mac software runs this word correction program that **never gets the right word.** In my comment above, there are three misspellings, generated by typos. The Mac picked “lie” over “like,” “if” over “of,” and “over” rather than “other.” In typing this entry, it changed “picke” (I left off the “d”) and changed it to “pike.” I retain the option of having misspellings underlined in red, but they rarely show up, because there are no misspellings – they’ve been changed to the correctly spelled wrong word. My comments are crazy enough without the word changes; really don’t need help from this random word corruption…

  18. jonas says:

    BR, I think you’re too harsh on narratives. If you redefine narratives to only mean the bad ones, then sure Nate Silver is not only selling narratives. But his explanatory style is a big part to his success. Most of his readers are not savvy enough to check Nate’s numbers/stats themselves.

    Also, qualitative judgments are not valueless. I think they complement just purely quantitative ones, which is also just a partial picture. The problem is that the listener has a tough time discriminating between the good ones and poor qualitative judgments. How do you judge, as you say?

    But if you look deeper, he/she has the same problem with quantitative judgments as well because they’re not good enough with math and stats. It’s just a newer field (conning the public at large with data), so those who game the system hasn’t figured out all the angles yet. The con artists will start small (conning key decision makers with data), and move up from there.

  19. [...] Beware of a good story – The Big Picture [...]

  20. [...] The Dangers of Non-Modeled Narrative Story Tellers (December 3rd, 2012) [...]