The following discussion is by Lakshman Achuthan and Anirvan Banerji, co-founders of ECRI:
~~~
There’s been much attention to our work of late, culminating in a report this Tuesday from Bank of America listing several critiques of our Weekly Leading Index (WLI). Apparently, the clinching argument is that “If a single indicator always accurately predicted the trajectory of the economy, the demand for Wall Street economists would be significantly reduced.”
To some Wall Street economists, it may seem self-evident that there should be strong demand for their views. But this is not at all clear to us as far as their recession-forecasting function is concerned. After all, a 63-country IMF study on economists’ recession-forecasting prowess concluded that “The record of failure to predict recessions is virtually unblemished.” In contrast, the IMF subsequently noted that ECRI “has actually had a very stellar record” of recession forecasting.
As we outlined in our 2004 book, Beating the Business Cycle, there is no Holy Grail of economic forecasting, even among our large array of state-of-the-art leading indexes, of which the WLI is but one. It seems that many of the self-styled experts on the WLI either haven’t read the book, or simply don’t understand the parts of the book where we repeatedly state, in no uncertain terms, that ECRI does not use models.
We aren’t entirely surprised. Virtually the entire analytical community has been trained to believe that any and all quantitative approaches to forecasting must involve models, i.e., simplified representations of reality. The idea that there could be rigorous quantitative approaches that are not model-based seems to be entirely beyond their ken. Their understanding of analytical techniques is so model-soaked that it reminds us of an insightful comment from psychologist Daryl Bem: “It takes a very intelligent and non-parochial fish to realize that his environment is wet.”
So it is with economists who cannot imagine that ECRI’s leading indexes are not model-driven or based on back-fitting of data. Thus the BofA report notes that “the ECRI (sic) and other leading indexes . . . fit the business cycle better ex post than ex ante” and that “This is an example of a broader issue in all statistical models of the economy. The data fit much better in-sample than out-of-sample.”
They just don’t get it. While this may be a valid criticism of statistical models of the economy, ECRI’s leading indexes are not, in any sense, statistical models of the economy.
BofA’s ignorance of the facts continues in their use of a 1993 (yes, 1993!) Dallas Fed study of the Commerce Department’s LEI to impugn the WLI. The Fed paper correctly points out that the-then LEI got revised a lot — a situation that hasn’t changed now that the Conference Board maintains the index. However, none of what the Fed reviewed in 1993 had anything to do with the WLI, which is the target of the current BofA report.
Furthermore (not that we’re fans of the Conference Board’s LEI) but the Dallas Fed study may not be valid in the first place, since the results are based on a Bayesian model chosen by the Dallas Fed to generate recession and recovery signals from the LEI. In other words, it isn’t clear whether failures highlighted in that report have to do with the LEI itself or the assumptions and the specifications of the Bayesian model used in that study.
Still, the BofA’s latest forecast of a “recovery, stronger than what we saw in the early 2000s and 1990s, but weaker than V-shaped recovery in the early 1980s” is very much in line with what ECRI predicted — in August 2009. At that time, we said the recovery would be “at a stronger pace than any the United States has seen since the early 1980s.”
BofA’s parroting of our forecast from late last summer reminds us of Jon Stewart’s segment on Nowcasting where experts describe something that’s already happening as though it’s coming (Jason Jones starts at three-minute mark).
Bottom line, neither the “experts” predicting that the sky is falling based on the WLI, nor the other “experts” indulging in misinformed WLI-bashing in an effort to discredit the super-bears, have a real clue to what the WLI is all about. We created the WLI not to be an infallible, stand-alone recession-forecasting machine, but as one small part of a much larger array of leading indexes (each made up of many economic indicators) — like the especially prescient U.S. Long Leading Index. This array amounts to a sophisticated sequential signaling system of the economy’s cyclical turning points. The WLI is designed to be interpreted in this broader context, and its message today is quite simple: A slowdown in U.S. economic growth is imminent, but a new recession is not.
>
Category: Data Analysis, Economy
Please use the comments to demonstrate your own ignorance, unfamiliarity with empirical data and lack of respect for scientific knowledge. Be sure to create straw men and argue against things I have neither said nor implied. If you could repeat previously discredited memes or steer the conversation into irrelevant, off topic discussions, it would be appreciated. Lastly, kindly forgo all civility in your discourse . . . you are, after all, anonymous.
You forgot to mention the Leaking Indicator – as in how stinky is it really going to get?
What, fact-based economics? Almost an oxymoron.
Saw Lakshman on a video yesterday and was impressed with him in contrast to almost every other economist I’ve ever seen. They generally remind me of the “Assume a can opener” joke, which, surprisingly, produces a ton of hits on Bing and Google.
ECRI has been way out in front throughout this crisis. I have noticed how some economists (think Rosenberg) have latched on to some of ECRI’s data to support their own (double dip) view of the world. I follow ECRI and still read Rosie but in the case of the latter I assign him about the same level of cred as I do Don Luskin.
I strongly recommend their book “Beating the Business Cycle”
http://www.businesscycle.com/resources/books/
1st, my apologies.
You wrote a piece here a few months ago and I responded with a few snarky comments because I didn’t know anything about your work. In error, I lumped it together with the flotsam of magic chartists and the sell side pitch work of your average sales pundit.
In fact, you appear to be making a sincere effort to create for economics something similar to what QC people and engineers routinely do using statistical process control … attempting to measure and explain normal and abnormal variation using graphical communications as your media. I would happily bookmark your site for further reference if timely free data were available. Being a poor bum, I only read free stuff.
Agree about econometrics. It is a laughable tool for projecting and serious estimation. As proof, all one needs to ask the average econometrician is “If you’re so smart then why aren’t you rich?” At least, why aren’t you rich from econometrics and not from sales of newsletters to rubes who think you know the future? This same logic applied to sales pundits on TV and predatory financial advisers who huckster their capabilities and earn most of their green from management fees.
Agree about the likely course of the future economy with one caveat. GDP measurements may or may not support this conclusion. However, since simple inventory rebuilds and massive government stimulus can make GDP appear to rise to superior levels, then all this means is we have another useless statistics coming from Uncle Stupid.
Without having the tools you use for your prognostications, I think we will see a reversion to a time where people live within their means because junk credit, junk investments, and nanny government have seriously damaged the economy. Unlike before the Y2K bubble, China will be in competition for jobs, sales, and resources. Thus a slack US economy will be the new normal until an educated, creative, and motivated US workforce does something creative and industrious (fat chance).
It’s an ill spill that blows no good! The BP stimulus
http://www.miamiherald.com/2010/06/22/1695167/spill-puts-unemployed-to-work.html
While good economic forecasting is usually helpful to provide contextual structure, I find the fundamental flaw to be the implicit linkage of anticipating sequential processes to lead to expected outcomes when one is also dealing with “randomized” events. These may manifest in the forms of Taleb’s Black Swans or obscured White Swans, which defy the rigid angularity of a predictable process.
Correction:
Above, I used the phrase “nanny government”.
Rather, I should have used a phrase that encapsulated corrupt, incompetent, industry owned, always on the wrong side (deregulation for the crooks, regulation for the industrious, free money for the crooks and slackers, etc), self-perpetuating, and getting worse.
Thank you for the correction! It’s not so much that we need less or more government–we desperately need competent government. Which is being provided by neither of the two big political parties.
[...] there was a ‘broo ha ha’ brewing ‘out there’ until we came across a ‘post’ on Ritholtz website. Lakshman is taking dead aim and BAML to task about BAMLs misunderstanding OF ECRI. Lakshman ALSO [...]
How difficult, actually, is forecasting a reversion to the mean? That’s essentially what these guys did with their oh-so presicient call that the recovery would be stronger than any since the ’80′s. Doesn’t the recession preceding the recovery being worse than any since the ’80′s almost guarantee that the recovery would be of like magnitude? It takes proper application of Baysian statistics to figure this out?
Now their forecasting method, which apparently takes account of all the measurable variables in the universe since it IS NOT A MODEL, predicts a lower level of growth, but no “double-dip” ice cream. This is a really hard prediction as well, considering that reversion-to-the-mean thingy.
Honestly, a decent astrologer could make predictions as prescient, but at least he’d give the credit to some unfathomable mystery and not claim to have devised a superior forecasting methodology based on scientific application and understanding of the relevant variables.
Here’s a prediction based on my reading of the stars in the heavens: The earth won’t end in 2012, but US economic growth rates will be intricately intertwined with prospects for Obama’s re-election, varying inversely with the degree of uncertainty in the outcome.
I think highly of the ECRI and of Mr. Achuthan. But I was truly disappointed in his “fluff” book, Beating the Business Cycle, other than his section on the leather industry and how that effected the inventory cycle. That was informative but it was only (if memory serves) about half a chapter. IMHO.
If ECRI is forecasting but claim they are not using a model then they are using a theory (or theoretical framework) to make predictions. Even if that theory only stipulates that the market behaves more or less lawfully (causal relationships) and will signal its next move via strongly correlated indicators there still must be an underlying set of assumptions that, in principle, could be wrong.
The fact that ECRI has a better record than most prognosticators (having referred to their key indicators for years I can certainly affirm that) and relies upon a large network of signals rather than a single set of relationships improves confidence that, even if the signal is incorrect, it is unlikely to be catastrophically so (being ‘wrong’ may not be too costly). That’s good, very good in fact, but it does not remove the long tail nor can it tell you how fat that tail is.
Welcome Laksham,
Love your work. I hope to see around The Big Picture more.
Having just turned the last page on The Black Swan (2nd edition) last night I say WHO CARES and ECRI will be found to be incorrect to. Especially if more people start relying upon them and their indicators to make financial decisions. It’s the fact that people tend to ingore them that is probably allowing them to be somewhat accurate at this point in time. They will be right, until they are wrong and they will be wrong eventually.
I think I agree with NNT, we should get rid of all the economists and their forecasts.
Mark Hulbert’s latest column makes an irrelevant attack on ECRI’s Leading Weekly Index by showing that a reading of minus 10 percent did not predict past recessions in advance:
“Double dip — or double take?”
http://www.marketwatch.com/story/is-a-double-dip-recession-certain-2010-06-22
The latest WLI reading is minus 5.7 percent, so you have to wonder why Hulbert used a threshold of minus 10 percent.
seneca: One of Rosie’s recent newsletters made a reference to -10% being an almost perfect indicator of a recession. His newsletters are chock full of data and free:
http://www.gluskinsheff.com/
Thanks for the post; any further official explanation or discussion of ECRI’s methodologies would be welcome.
I find it rather incredible that given the attention ECRI’s work receives, there is such a misinterpretation of its work and effectiveness. Perhaps part of this misinterpretation / misunderstanding is because it is proprietary (i.e. a “black box” situation with little documentation or information provided).
However, I ascribe a lot of the misunderstanding to a lack of proper due diligence and poor / lack of analysis by the professional community.
Mike in Nola: Thanks. David Rosenberg is right. Using the table in the aforementioned Mark Hulbert column, since 1969 the ECRI Leading Weekly Index got as low as -10% on five occasions. Each time, the economy was already in recession. But since the Leading Weekly Index is currently -5.7%, Rosenberg’s observation is nice to know but not all that germane.
[...] Picture post: ECRI Weekly Leading Indicators Widely Misunderstood. [...]
I think one of the misconceptions about what ECRI is doing is that they are calculating an indicator which *signals* impending recessions and recoveries, rather than building a forecasting model which *predicts* the nature of such periods. I don’t believe I’ve ever seen them predict the quantitative nature of any recession or recovery. They have only written that one is coming, and qualitatively how strong or weak it may be.
To RW’s point, if one considers the compilation and calculation of an economic indicator as equivalent to a theory, then so be it. I don’t equate the two myself. I would say that what ECRI has done is simply to devise an economic measuring stick. They do not purport to know precisely why the measuring stick works, except perhaps in internal documents and discussions.
In any case, their indicators have been vastly more successful than anyone else’s model. I have to wonder then, what’s the big deal about whether it’s a model or not? It works!
I found this analysis of the ECRI indicator intriguing…
[...] that ECRI’s weekly leading index supports a double dip. Everyone, that is, except for ECRI themselves. Hussman and others argue from a sample set of 7 recessions, and while I do not disagree with using [...]
[...] we met, ECRI was explaining how their weekly leading indicator was being used to justify positions and forecasts that were [...]
[...] really leading, or are they misleading indicators? An IMF study of 63 countries said that (quoted here) “The record of failure to predict recessions is virtually [...]