Fun With Data Analysis: The Art of the Plausible

Barron’s Mike Santoli looks at one of our favorite pet peeves: The use and abuse of data. His column this week, The Limits of History, notes that this has become especially prevalent of late:

"People who try to handicap the markets for a living practice the art of the plausible. Many trudge from conference room to lunch table to banquet hall lugging PowerPoint decks full of unobjectionable statistical touchstones for commission-wielding clients. At times of investor confusion and market dissonance, such as now, their art is often reduced to carving a slice out of economic history that ratifies their existing outlook."

Mike then proceeds to look at a "dog’s breakfast of the kinds of historical analogies making the rounds." What is especially amusing about his list is the number of "Every time X happens, Y has occurred" that collectively produce all manners of mutually exclusive results. Since "X" occurred we will definitely have/avoid a recession; Stocks are undervalued/overvalued; Markets must rally/fall.

What is an investor to do? Whenever you are confronted with an "incontrovertible proof" based on historical data, prior to taking any action, I suggest asking yourself this short list of questions:

• Do we have enough historical examples? Is the data sample statistically significant?

Causation or Correlation? Does "X" cause "Y" to occur? Or, are we presented with two   things that may have the same underlying causes? Is there even interaction between X & Y?

Coincidence? How possible is it that these two items are utterly unrelated (i.e., proof-we-are-clueless Superbowl indicator).

• Look for differentiating elements in different time periods: What factors are similar?  What factors are different?

Compare interest rates, inflation, dividend yield, P/E contraction or expansion, sentiment, overall market trend, business cycles — across different eras. Might that account for potentially different outcomes?

• Any recent market environmental changes (regulation shift, financial innovation, etc.) have an impact? What might these specific changes do to the data? Consider Decimalization, ETFs, online trading, change in dividend tax, etc.

Subjective versus Objective measures: Are the factors under discussion hard numerical data, squishy or somewhere in between?  Percentage of stocks over 200 day moving average is objective; I find some chart pattern readings subjective. Earnings at time have been rather subjective; official inflation measures somewhere in the middle. 

• Consider things in terms of probabilities, not outcomes: Assume a causative factor resulted in a specific event (X –> Y) 7 out of 9 times. The most you can say is that when "X" occurred in the past, it has resulted in "Y" approximately 78% of the time. 

• There is a difference between historical occurrence and future likelihood. In the example above, this does not necessarily even mean that since "X" has just occurred, there is a 78% that "Y" will happen. Consider: was the first X/Y occurrence really a 100% or zero? Did the second one become 100% or 50%, then next a 66% or 33%?

Contextualize data: Sometimes a single data point — even a mean or median — only tells half a story. Any data point can be trending or reversing. Going higher, lower, topping, bottoming. Each of these may have differing implications for what comes next. Inflation is high, but coming down. Gold is high — and going up. It helps to think of data not as a still photograph, but as a frame in an ongoing film.

I’m sure there are more — that short  list is off the top of my head.

Any others? Please make your suggestions below. If we get enough good ones, I’ll try to massage this into a more formal column.


The Limits of History
Mike Santoli
Barron’s, MONDAY, JANUARY 28, 2008

Category: Apprenticed Investor, Data Analysis, Markets, Mathematics, Quantitative

NYTimes, Société Générale, & Me

Category: Corporate Management, Credit, Derivatives, Federal Reserve, Media

The Bulls, The Bears, & the Media

Category: Economy, Financial Press, Media, Psychology, Weblogs

Black Friday Night Jazz



On days like this, where the market opens up over 100 and closes down 170, I always get that "Black Friday" feeling — that no one really wants to carry much equity exposure over the weekend.

On rare occasions, these days set up ugly Mondays (Wait! Didn’t we just have one of those?)


But its Friday night — Enough market talk! Its time for some jazz to mellow out to.

I’m a big Steely Dan fan. Saw ‘em live a few times, always loved this song.

Read More

Category: Digital Media, Friday Night Jazz, Music

Conspiracy theory of the day

Category: Derivatives, Trading

Quote of the Day: Ackman on MBIA

Category: Corporate Management, Credit, Derivatives

Developing Economies Face Reckoning

Category: Markets

Read it here first: Fed Responding to Stocks?

Category: Federal Reserve, Markets

Santelli vs Cramer

Category: Financial Press

Fed: We didn’t know about SocGen trades

Category: Federal Reserve, Psychology, Trading