"All models are wrong; some are useful."
-George P. Box
I enjoy intelligent criticism. It forces you to tighten up your analysis, more aggressively dig up facts, communicate your position more effectively. Good criticism should be challenging, eloquent, and raise the argument bar — making you sharpen your rhetorical pencil.
Then, we have the criticism that can be most charitably described as "wanting." These come in a variety of forms, ranging from the simple ad hominem attack to the complex Straw Man arguments, to the classic taking quotes out of context.
There are a many other weak, disreputable forms of argument, and they all find a ready home in the blogosphere.
Lately, I have been noticing an increase in the conspiracy accusation. In particular, our many discussions on the systemic understatement of Inflation or Unemployment gets derided as "There goes Ritholtz again with his conspiracy theories!" This one statement manages to combine all 3 rhetorical errors mentioned above above into one lazy and dishonest argument.
Today, we will spill a few pixels on this, and what it means.
Let’s begin with how BLS models are constructed and
function in the real world. I find this complex subject of to be quite fascinating. The entire statistical output from the various governmental data bureaus is like a giant crossword puzzle, Sudoku and murder mystery wrapped up in one. Dissecting the publicly available data, teasing out nuance, placing the snapshot into the context of the larger moving picture is a delightful intellectual exercise.
I can understand that these sorts of puzzles are not for everyone, but I simply cannot fathom how they are so easily dismissed. Rather than
address the gravamen of these intriguing mathematical puzzles, they are simply waved away. Conspiracy
theory! Tin Foil Hats! Nut Jobs!
I began this post with a little known quote, which I will slightly modify "All models are flawed, but not worthless." (
I misplaced the author’s name Thanks!).
The issue is not whether any specific model might be wrong: ALL MODELS ARE BY DEFINITION WRONG. They are approximations, a numerical depiction of a small portion of universe. The true question for interested statisticians, mathematicians and economists is "just how wrong are they?" This is not, as some have implied, a question of nefarious cabals or governmental conspiracies. It is simply a mathematical reality.
For example, we know that the way Inflation is measured has changed
over the years. The focus on the core rate, variable weighting of items in the CPI basket,
the report and changes from the Boskin Commission
– these all end up altering the output of the inflation data. All of
the changes to the BLS inflation methodology are released by BLS,
painstakingly footnoted, and very, very public.
An even better example is the BLS expansion of the Birth Death
model. To grossly oversimplify, BLS Statisticians identified a shortcoming in the way the Establishment Survey picks up new jobs from start up firms. Rather than
interview people at home (like the CPS study which determines the
Unemployment rate does), the CES takes data from employment tax filings
at existing companies. However, economists know that much of the new job
creation int he US comes from new companies. In order to capture the jobs
currently missed by the old CES methodlogy, the BLS expanded a B/D model. Now, they use new
state incorporation filings, and a formula that deduces how many new
jobs go along with these newly born firms. My criticisms about this are legion, but none claims this is a hidden conspiracy or political
manipulation by the BLS. The changes to the Birth Death model are all very public, and well documented. Its just that I believe they are flawed — and those flaws seem to all have a bias to systemic make things look better than they really are.
All of the various economic models used by Bureau of Economic Analysis, and the inputs and algorithms involved, change the output that gets published as your favorite monthly data point.
As I have said repeatedly over the years, all of the data is there — its released in a variety of forms, raw, non-seasonally adjusted, monthly, year-over-year. You just have to fight your way through it. Here’s what we said on the subject last year:
"One of the things people do not understand is a basic truth about the BLS, BEA, Commerce Department, and other government data sources. I get constant emails warning me of the dark cabals manipulating the official releases.
While I have been critical over the years of the models and
methodologies employed, I do not believe this is any grand conspiracy.
Sure, the headlines are often misleadingly spun, but the data is all
there for whoever wants to peruse it. Indeed, if you strap on your
green visor, you can find all of the data releases for Inflation,
Unemployment, GDP, New Home Sales, Durable Goods, etc. The
non-seasonally adjusted, non-hedonics, no substitutions data is all
right there. You need only bring a critical eye and a dollop of
skepticism, and you can usually deduce the real meaning beneath the
That is my long held view. You are free to disagree with it for any number of reasons. You might recognize the error, but think it is so small as to be meaningless. Perhaps, you think there is a conspiracy. Maybe you believe the BLS models — unlike every other model ever created by mankind — are flawless.
However, if your intellectual sophistication and analytical acumen is at a level where you find it easier to claim that those who disagree with your views on inflation, employment and GDP wear tin foil hats and consort in Area 51, well that’s just so much intellectual detritus, and I am calling you out on it.
Your dismissive comments are lacking in critical rigor. They are analytically vapid, and intellectually lazy. This is my pushback to the creeping anti-intellectualism that has been working its way into what is usually considered a somewhat thoughtful and analytical discipline: Market analysis and economics.
There is a silver lining: They put the reader on notice that they are dealing with an intellectual lightweight, a political hack, someone not worthy of any further expenditure of your limited time and finite attention.
Its one thing to see the comments on other blogs — I expect no less from anonymous cowards; However, I refuse to tolerate this sort of nonsense on actual blog posts by authors who should better.
No Conspiracy Theory — Just Data (November 2007) http://bigpicture.typepad.com/comments/2007/11/no-conspiracy-t.html
Other Worthwhile Discussions:
Does the Government Understate Inflation? http://www.fundmasteryblog.com/2008/06/17/does-the-government-understate-inflation/
Why Bears Always Have the Best Arguments http://paul.kedrosky.com/archives/2007/12/13/why_bears_alway.html
Merrill’s David Rosenberg goes postal on the inflation conspiracy theorists
Statistics as a government plot
Area 51 coment
Hall of Shame:
SCOOPED BY MUCKDOG
Thinking About Butterflies Randomosity
Role of proprietary trading at banks http://finaxyz.blogspot.com/2008/01/role-of-proprietary-trading-at-banks.html
How To Understand Inflation Data http://seekingalpha.com/article/24572-how-to-understand-inflation-data
When making money becomes too easy…….
The Conspiracy to Keep You Poor and Stupid
Government leaders and oil producers will meet in Saudi Arabia over the weekend to discuss record crude prices. Saudi Arabia, the world’s biggest oil producer, is considering raising production. But will this solve the problem?
click for video
Exploding commodity prices, lax monetary policy, and sovereign wealth funds
20 June 2008
Leaders, Oil Producers to Meet to Discuss Supply, Prices: Video
Bloomberg, June 19, 2008 23:12 EDT