Posts filed under “Analysts”
The Failure to Forecast the Great Recession
November 25, 2011
Experience shows that what happens is always the thing against which one has not made provision in advance.
– John Maynard Keynes1
Our best plan is to plan for constant change and the potential for instability, and to recognize that the threats will constantly be changing in ways we cannot predict or fully understand.
– Timothy Geithner2
The economics profession has been appropriately criticized for its failure to forecast the large fall in U.S. house prices and the subsequent propagation first into an unprecedented financial crisis and then into the Great Recession. In this post, I examine the performance of the forecasts produced by the economic research staff of the Federal Reserve Bank of New York (New York Fed) over the period 2007-10 and consider some of the reasons why we, like most private sector forecasters, failed to predict the Great Recession. This spreadsheet contains staff forecasts, the outcomes, and a standard measure of private sector forecasts—the Blue Chip consensus. In addition, staff material prepared for bi-annual meetings of the New York Fed Economic Advisory Panel provide some further insights into the evolution of the staff outlook.
The staff forecasts of real activity (unemployment and real GDP growth) for 2008-09 had unusually large forecast errors relative to the forecasts’ historical performance, while the forecasts for inflation were in line with past performance. Moreover, although the risks to the staff outlook were to the downside throughout this period, it wasn’t until fall 2008 that a recession as deep as the Great Recession was given more than 15 percent weight in the staff assessment.
How Bad Were the Forecasts for Real Activity?
Economic forecasters never expect to predict precisely. One way of measuring the accuracy of their forecasts is against previous forecast errors. When judged by forecast error performance metrics from the macroeconomic quiescent period that many economists have labeled the Great Moderation, the New York Fed research staff forecasts, as well as most private sector forecasts for real activity before the Great Recession, look unusually far off the mark.
One source for such metrics is a paper by Reifschneider and Tulip (2007). They analyzed the forecast error performance of a range of public and private forecasters over 1986 to 2006 (that is, roughly the period that most economists associate with the Great Moderation in the United States).
On the basis of their analysis, one could have expected that an October 2007 forecast of real GDP growth for 2008 would be within 1.3 percentage points of the actual outcome 70 percent of the time. The New York Fed staff forecast at that time was for growth of 2.6 percent in 2008. Based on the forecast of 2.6 percent and the size of forecast errors over the Great Moderation period, one would have expected that 70 percent of the time, actual growth would be within the 1.3 to 3.9 percent range. The current estimate of actual growth in 2008 is -3.3 percent, indicating that our forecast was off by 5.9 percentage points.
Using a similar approach to Reifschneider and Tulip but including forecast errors for 2007, one would have expected that 70 percent of the time the unemployment rate in the fourth quarter of 2009 should have been within 0.7 percentage point of a forecast made in April 2008. The actual forecast error was 4.4 percentage points, equivalent to an unexpected increase of over 6 million in the number of unemployed workers. Under the erroneous assumption that the 70 percent projection error band was based on a normal distribution, this would have been a 6 standard deviation error, a very unlikely occurrence indeed.
Did We Calibrate the Risks to the Forecast Appropriately?
Of course, there is much more to forecasting than the point forecasts reported in the spreadsheet. In particular, it is crucial to assess the uncertainty and risks around any point forecast.
Throughout this period, the uncertainty and downside risks assessed around our point forecast were substantial relative to economic fluctuations in the Great Moderation. One way of gauging the appropriate calibration of downside risk is to measure the depth of the implied recession if the risks were realized.
The chart below does this by considering the probability distribution of the four consecutive quarters with the lowest GDP growth in a recession. It presents results based on the staff outlook in April 2008 and November 2008. The depth of the mild recessions shown in the chart was typical of the type of recessions expected during the Great Moderation. The actual depth of the 2007-09 recession as gauged by this metric is currently estimated to be 5 percent. As the chart shows, it was only by November 2008 that the probability of the actual outcome was above 15 percent.
Upon seeing this type of calculation, Robert Barro, a Harvard professor and member of the New York Fed Economic Advisory Panel, noted that the decline in real stock market values in the United States was similar to that observed in countries experiencing depressions. Taking this relationship into account in the calibration of the downside risks produced about a 50/50 chance of the currently observed depth of the Great Recession (see the March 2009 Wall Street Journal op-ed article by Barro for his assessment of the probability of a depression).
Recent Forecast Performance
In contrast, the New York Fed staff forecasts for 2010 made in 2009 and early 2010 are quite accurate (under the assumption of no major revisions to the estimates of GDP growth in 2010). This accuracy, however, has not been sustained through 2011. As widely discussed by a number of Federal Reserve officials, the level of real activity in 2011 has been disappointing relative to expectations. This shortfall is evident in the chart below, which compares forecasts for GDP growth in 2011 and 2012 produced in April and October 2011. However, this chart also depicts the uncertainty and risks around the staff forecast as of April 2011. Given the uncertainty around the April forecast, the subsequent changes to the outlook are not very surprising. On the other hand, near-term downside risks to this forecast were low compared to other forecasts produced in the last four years, so the direction of the change was more surprising.
Why Did We Fail to Forecast the Great Recession?
The quotations from Keynes and Geithner at the start of this post capture the importance of constantly striving to ensure that policy is robust to unexpected events. As explained in much of the recent work of the 2011 Nobel Prize–winning economist Tom Sargent, the unexpected events for which policymakers need to make provision have the characteristic of being the most likely unlikely bad event. The collapse in housing prices and its propagation to the economy certainly fit this description.
A leading example of how effective a robust approach to policymaking can be is the 2009 Supervisory Capital Assessment Program. In this program, large U.S. banks were evaluated against a capital standard under the assumption of a longer and deeper recession than contemplated in the prevailing consensus estimate. The idea was that if banks had sufficient capital to continue performing their intermediation function under this more adverse scenario, the scenario was less likely to occur.
Indeed, in the period leading up to the financial crisis, analysts who were suspicious of the stability of the Great Moderation, such as Nouriel Roubini, offered assessments that proved to be significantly more accurate than the point forecasts of New York Fed research staff or most professional forecasters in gauging the potential for unlikely bad outcomes. On a more positive note, if one compares the downside risk in the New York Fed research staff outlook with that of the Survey of Professional Forecasters (see the chart below from April 2008), there is some evidence that we had a more sober assessment of the risk of a severe downturn than did private sector forecasters.
Looking through our briefing materials and other sources such as New York Fed staff reports reveals that the Bank’s economic research staff, like most other economists, were behind the curve as the financial crisis developed, even though many of our economists made important contributions to the understanding of the crisis. Three main failures in our real-time forecasting stand out:
- Misunderstanding of the housing boom. Staff analysis of the increase in house prices did not find convincing evidence of overvaluation (see, for example, McCarthy and Peach  and Himmelberg, Mayer, and Sinai ). Thus, we downplayed the risk of a substantial fall in house prices. A robust approach would have put the bar much lower than convincing evidence.
- A lack of analysis of the rapid growth of new forms of mortgage finance. Here the reliance on the assumption of efficient markets appears to have dulled our awareness of many of the risks building in financial markets in 2005-07. However, a March 2008 New York Fed staff report by Ashcraft and Schuermann provided a detailed analysis of how incentives were misaligned throughout the securitization process of subprime mortgages—meaning that the market was not functioning efficiently.
- Insufficient weight given to the powerful adverse feedback loops between the financial system and the real economy. Despite a good understanding of the risk of a financial crisis from mid-2007 onward, we were unable to fully connect the dots to real activity until 2008. Eventually, by building on the insights of Adrian and Shin (2008), we gained a better grasp of the power of these feedback loops.
However, the biggest failure was the complacency resulting from the apparent ease of maintaining financial and economic stability during the Great Moderation. Perhaps most important, as noted by some analysts as early as the 1990s, these adverse consequences of the Great Moderation were most likely to arise from the actions, judgments, and decisions of financial market participants:
Longer stretches of economic growth imply greater leverage and complacency and thus, greater financial problems when recessions do occur.
–William Dudley and Edward McKelvey3
1Letter to Jacob Viner, June 9, 1943, Collected Writings of John Maynard Keynes, ed. Donald Moggridge, vol. 25. London: Macmillan, 1980.
2Letter from the Chair, Financial Stability Oversight Council Annual Report.
3The Brave New Business Cycle: No Recession in Sight, Goldman Sachs Economic Research Group, January 1997.
Interesting quote from Laszlo Birinyi on Wall Street research: “One of the dark secrets of the market is we don’t really do much research on Wall Street. I started off on the trading desk. I worked at my job. There were a lot of people who really hadn’t done the work, and what they were…Read More
> Stock investors may take days to distinguish real news from noise, according to Federal Reserve Bank of New York. This is especially true these days, given false announcements of bailouts, Fed interventions and rescues. They tend to cause fake short squeezes that temporarily spike markets, only to see them ultimately head lower. To get…Read More
Morgan Stanley Research channels our prior discussion on earnings during recessions: “We have heard investors suggest $80 in EPS was a fair bear case for 2012. We decided to look at history as a guide in assessing the bear case EPS. The 2001 recession saw a 13% revenue decline and a 57% EPS drawdown. The…Read More
From Andrew Horowitz of the (The Disciplined Investor:
Last week I attended the Bloomberg Markets 50 Summit in New York. The setting for the event was the transformed Great Hall of the Community House at St. Bartholomew’s Church. The room was full of “jackets and ties” from all of the major brokerages, hedge funds and others involved in the fine art of investing.
The Bloomberg Staff were more than accommodating, friendly and informed. Everything was on a tight schedule as the event was being televised, so timing was to the second for the start at 9:55am. First a few words from Dan Doctoroff, president and CEO of Bloomberg to start off the morning and then he introduced the moderator and first panel of speakers.
The even was structured as a panel discussion, where the various speakers were comfortably seated on a lush white couches. Each panel had a topic and the moderators would ask for their insights on a specific topic. Overall the day was full of excellent and topical commentary and opinion focused on items that ranged from the European Crisis to Hedge Funds.
I was able to get some one-on-one time with Carson Block of Muddy Waters to discuss some of his recent findings in China. Nassim Taleb, famed author of Black Swans was not so kind and only could spend a minute or two. Stephen Roach, who just about tells you that he is always right, spared some time to talk with me about the rampant food inflation in Asia. I congratulated the John Chambers, the Managing Director and Chairman of Standard & Poor’s Global Sovereign Rating Committee for the work they are now doing in keeping the world’s government’s honest. We spoke about the continuing problems and specifically addressed the outlook for France. I asked about the recent AAA rating and what is the outlook. Of course he could not provide specifics, but mentioned that everything is up for review and nothing is permanent. I got the feeling that there is more to this story…
After the formal discussion/panel with Gary Gensler, Chairman of the Commodity Futures Trading Commission, I was able to have a few moments to ask him about the oversight of the CME. In particular, I questioned him about the recent plunge in Gold and Silver prices for no apparent reason, when later that same day a margin hike was announced. Was this leaked and is the CFTC looking at these? He replied that he was unfamiliar with the specific situation that I was referring to (was he kidding I thought?) but that they are “more interested” with leaks of government data prior to the official release. Take a listen to the actual recording of that conversation – HERE.
As for an overview of the day, it was interesting to see that there was a high level of disapproval of what the White House and Congress has been up to. That makes sense as this was a group of business and investment pros and they are in the cross-hairs of the government’s ambitious business-unfriendly programs.There was also a rather palpable negativity about the U.S. equity markets due to the current financial crisis unfolding in Europe. While there were a few panelists that had some upbeat comments, overall there was a lack of bullishness that I had expected from this group.
What follow are the notes that I took during the day. These are in no way a complete transcript of the panel discussions, but provide highlights of what I believed were the important points.
9:00am – 9:40am -Bernanke’s Balancing Act
Bernanke may not have much that he can do and that getting the committee to move will be a process. The next meeting with surely give us TWIST and then SHOUT, which is a more vocal communication. The FED is on the move it has a lot of problems in the economy it is dealing with. Expect more, not less in the future
The economy is clear. What should be done… Fed needs to be accommodative, but perhaps temporize too much, The big problem is that the transmission mechanism, is dysfunctional. That is typical after this kind of banking crisis. Better to get something that gets under the car and fixed the transmission The most evident is the need to get the banks lending. Region banks have not been as easy going as they would like to be. Maybe the foot soldiers should be listening to the Generals more closely. Somehow we need to get the banks to lend
Glen Hubbard: Fiscal or? Bernanke policies are limited. It is a fiscal need at this point. Investment has been slowed down due to regulations, no housing boom and the markets. there is a clear need to have a fiscal policy that allows for the benefit to
When asked; Is Fed trying to boost stocks? Mr. Doll replied that Fed is trying to install confidence. We need to get out of this confidence bear that we are in That is why in Jackson Hole he tried to extend the confidence by putting in a long term interest rate assumption so that some of the questions are cleared up.
General discussion about whether it is fiscal or monetary policy, generally all see that that there is a combined effort and that interest rates have low enough at this point. We are in a time that there is a slippery slope if we hope that Congress comes in and does what they have to. Communications tools need to be used and can me quite powerful as Congress and the Administration is impotent. All appear to agree that we are in need of stimulus and help.
Perhaps the biggest issue is the lack of leadership and the inability for them to lead. The most concerning is that there is also a crisis of confidence that started and is continuing. It is the combination of the lack of leadership as well as the concern that the FED is not doing all it can (?really). The things that are being down are not helping. There is a general agreement that the plans and other stimulus measures have not done anything and are not going to do much in its current forms. If nothing is done on the fiscal side, there will be a drag of about 2% of GDP.
Demand is weak as there is no confidence but also because there is not enough stimulus on a short term basis. There are things that can be done, but are not. The overall political situation at this time is at best, un-encouraging.
9:40am – 10:20am – Euro Breakup or The United States of Europe
Discussion about the the coordinated action and that is the coordinated action is the news This is perhaps why Juergen Stark resigned last week. This is a really big deal. It also calls into question of depth of the problem. In other words, is the situation so deep that there needed to be a coordinated action. This is clearly a liquidity problem.
Austerity is supported by governments but the evidence is that Austerity has failed as confidence is collapsed, growth has failed and there is discontent. Riots and a great deal of increased unemployment has shows that there has been the wrong path followed.
Is there a way to have an orderly default of a sovereign nation? If Greece can leave the Euro quickly is unlikely. There are basis problems as little as the inability for Greece to transact. There will be major lawsuits etc. Leaving the Euro will be unlikely for Greece in the short term. Default is more likely and that may need to be done.
When you have banks that are unable to get funds and in particular US dollars that is a problem. Banks have been seeing a liquidity problem and the amount f swap lines between the US and Europe and therefore it is already available. For right now, this was a necessary liquidity problem, though it is not a final solution. The action today was a short term fix to somewhat recapitalize the banks, for a period. Perhaps months. There needs to be another mechanism to bring these banks up to the Basel standards.
What needs to happen next? Has the ECB overstepped it bounds.
The ECB is going to have to do something and the move today was helping the markets short term, and in the end they may look to put together treaties. In the near future they need to cut rates to help confidence. In the next few months the ECB etc will need to buy Spanish bonds as well as other over the next couple of years. The Euro is under pressure but no one can afford a breakup of the Euro. Any country that will leave would be hurt terribly. Asia could not afford to have a breakup of the Euro or the EuroZone. This is a long-term project to save the area and this will take years.
There is no growth in EuroZone and the fiscal austerity needs to be done in individual countries an then the stimulus for area can
John R. Taylor
This does not get at any of the major problems underlying the situation. Just yesterday there was a few of the banks that could not get funding. So the was more a requirement.
With the recent coordinated action in Europe, what to do? STAY AWAY. This is a horrifically poisonous environment. If your strategies have to be in Europe, it is difficult as we really have not grappled with the true solution. They cannot create growth if there is austerity and no growth. The Swiss will be able to do well with the caveat that the Swiss believe that Europe will solve their problems, but that is unlikely.
10:20am – 10:40am – A Conversion with Ray Dalio
If it was not unexpected that we saw this last crisis, what is next? The biggest problems is that we are not having a conversation at the highest level that discusses the machine. Even though that the we make decisions, we need to know how the economic machine works. We need to have a quality conversation about how the economic and political machines work and then we can create better decisions.
Even though there will be choices made, even after choices are made, they may not be the best for everyone. For example lets imagine that you earn $100,000 per year and have no debt. You can go to a bank and get $10,000 loan. That is okay and you can spend and then if you do it over and over again and then there will be a time that you can no longer pay the debt service. Then you can lower interest rates and when there is no more room there… problems.
Credit can be created in countries and the same cycle as above will persist. How much of the money that is being spent by ECB to do a restructuring and other measures to fix things. One of the most important things now is to understand what is going on and make some important plans, rather than waking up every day to a new surprise.
On printing money: It is quite a handy thing to have. If you are a creditor with a linked policy then you have problems in that there are extremes that are created. For example Greece is a creditor and they cannot adjust and they are their economy is going to crash, if on the other hand you have those that are linked and needs to print then you have a bubble like China.
What is working for Bridgewater? Dalio says the he writes the daily so that he knows that he knows what he is doing wrong. If you diversify and don’t rely on only one things to make your year, but many ideas go into the Ray Dalio code and his template for the economy is available to download HERE. (pdf).
On Friday the Federal Reserve released its quarterly Flow of Funds data, current through June 2011. One of the more popular headlines from this data concerns the record amount of “cash on the sidelines“. Through Q2 2011, nonfarm nonfinancial corporate businesses held $2.05 trillion in liquid assets on their balance sheets. As the argument goes,…Read More
Italy Downgraded by S& P S&P just downgraded Italy’s credit. I’ve been warning about Italy since 2008. The Problem Will Spread Because NONE of the Fundamental Problems Have Been Fixed None of the fundamental economic problems in Italy or Europe or anywhere else have been addressed … let alone fixed. So the problem will only…Read More
> My Sunday Business Washington Post column is out. This morning, we look at earnings, analysts forecasting track record, and what that means for stock valuations if we have a recession. I actually like both headlines today: The online version is Market action a ‘conversation’ between investors very much sums up a philosophical view I…Read More
“Wow, that was a sobering meeting.” -Western Asset Management, Stephen Walsh, CIO > Did S&P leak US credit downgrade info to specific bond firms in advance of it occurring? That is the subject of a WSJ article today, and should be a question the SEC is investigating: “Standard & Poor’s Corp. officials held private meetings…Read More