Holy Snikes, this is HUGE, from Mike Konczal:

“In 2010, economists Carmen Reinhart and Kenneth Rogoff released a paper, “Growth in a Time of Debt.” Their “main result is that…median growth rates for countries with public debt over 90 percent of GDP are roughly one percent lower than otherwise; average (mean) growth rates are several percent lower.” Countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate, in fact.

This has been one of the most cited stats in the public debate during the Great Recession. Paul Ryan’s Path to Prosperity budget states their study “found conclusive empirical evidence that [debt] exceeding 90 percent of the economy has a significant negative effect on economic growth.” The Washington Post editorial board takes it as an economic consensus view, stating thatdebt-to-GDP could keep rising — and stick dangerously near the 90 percent mark that economists regard as a threat to sustainable economic growth.”

Is it conclusive? One response has been to argue that the causation is backwards, or that slower growth leads to higher debt-to-GDP ratios. Josh Bivens and John Irons made this case at the Economic Policy Institute. But this assumes that the data is correct. From the beginning there have been complaints that Reinhart and Rogoff weren’t releasing the data for their results (e.g. Dean Baker). I knew of several people trying to replicate the results who were bumping into walls left and right – it couldn’t be done.”

The three criticisms of Reinhart and Rogoff:

1. They selectively exclude years of high debt and average growth.
2. They use a debatable method to weight the countries.
3. A coding error that excludes high-debt and average-growth countries.

All of these bias the results towards their conclusion. If this is verified, it will be the biggest academic snafu since Professor Jeremy Siegel messed up his book Stocks for the Long Run relying on bad data.

This does not justify running huge deficits, but it also removes all of the urgency of the Austerity camp. A much more slow form of de-leveraging – what Ray Dalio calls “the Beautiful de-leveraging” — and not austerity is what appears to be what is called for.

I am watching this closely . . .

 

UPDATE:  April 16, 2013 4:02pm

Reinhart and Rogoff respond here.

 

 

Previously:
Jeremy Siegel is not having a good year (July 11th, 2009)

Sources:
Researchers Finally Replicated , and There Are Serious Problems
Mike Konczal
Next New Deal, April 16, 2013

http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems

Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogo ff
Thomas Herndon | Michael Ash | Robert Pollin |
Political Economy Research Institute 4/15/2013

http://www.peri.umass.edu/236/hash/31e2ff374b6377b2ddec04deaa6388b1/publication/566/

Category: Credit, Philosophy, Really, really bad calls

Please use the comments to demonstrate your own ignorance, unfamiliarity with empirical data and lack of respect for scientific knowledge. Be sure to create straw men and argue against things I have neither said nor implied. If you could repeat previously discredited memes or steer the conversation into irrelevant, off topic discussions, it would be appreciated. Lastly, kindly forgo all civility in your discourse . . . you are, after all, anonymous.

35 Responses to “Did Reinhart-Rogoff Screw Up Their Debt Research?”

  1. gman says:

    They did a great job on this paper. It fulfilled the goal of its sponsor. Pete Peterson got the result he paid for.

  2. rd says:

    The Austerity Club is only one step removed from the Goldbugs and Death Penalty As a Deterrent to Crime Supporters. They don’t need data to prove their theory as their theory is largely a morality viewpoint, not economic or statistical.

  3. DeDude says:

    A large number of red flags hit the scientifically minded (fact-addicted) person at first glance of the Reinhart Rogoff study.

    Two things that correlate can either have/or not have causative relationship from one to the other. Because of the lack of true controlled experimentation it is very hard to prove causation, but an unbiased approach would try to make the case both ways (1 causes 2 and 2 causes 1) – then argue why a specific causative direction was more likely then the other.

    The are dealing with data that is continuous yet they choose to analyze it categorically. Why choose 90% rather than 85% or 95%. A natural round number would have been 100%, so the fact that they use 90% smells like someone having tried a number of different cut-off values until they got the “correct” result.

    Not releasing all the data and all the code from your analysis so others can see what exactly you have done and repeat the analysis using other assumptions and cut-off values always should rise the question of what it is the authors fear.

    • papicek says:

      Not releasing all the data and all the
      code from your analysis so others can see what exactly you have
      done and repeat the analysis using other assumptions and cut-off
      values always should rise the question of what it is the authors
      fear.

      Bingo. Took the words right out of my
      mouth.

  4. ComradeAnon says:

    Data? We don’t need no stinking data!

  5. NMR says:

    Oh boy, if correct, Krugman is going to have a field day.

  6. VennData says:

    Hopefully, Awryhart and Wrongoff will apologize, give back all their speaking fees and possibly, hopefully, please tone down their right wing arrogance.

    • VennData says:

      …and what about the sales of the books? Nice job Princeton University Press. I’ll make sure to buy your next scathing critique of global Socialism in pre-sales.

  7. What?! Economists getting the wrong answers?! Economists disagreeing with other economists’ conclusions?! We’ve NEVER seen this happen!

    Let’s shed the ideological blinders and just do basic accounting: High government debt (by itself) just means that a nation’s people are going to have to pay taxes in order for “their” government to pay interest to those who issued the credit to the government. I think what’s critical to how this impacts growth is who the creditors are, how much interest is involved, and what the creditors do with the interest received… versus what those workers would’ve done if they didn’t have to pay taxes to pay interest. It’s easy to see that a lot of the time this would be a bad situation for a country to be in. Especially if NEITHER the creditors nor the workers are able to do anything to generate growth to relieve the debt burden.

    I think the whole debate has more to do with the fact that everyone and their cousin knows damn well that there’s a lot of pain ahead, but some folks on both sides are cravenly trying to foist the problem off on someone else, instead of pulling together and getting everyone to do their share. (It doesn’t help that in the absence of either honest accounting or political leadership, no one can agree on “fair share”…)

    P.S. I caught that mention of Ray “Beautiful Deleveraging” Dalio… doesn’t he also own gold, and widely publicize the fact that he owns it, during a high-profile trip to Davos? Maybe because “beautiful deleveragings” are a lot like arranged marriages (or a box of chocolates), and you just never really know what you’re going to get?

    • DeDude says:

      I agree that a more relevant analysis has to include who get the benefits/pain of the debt and who suffer/gain from the change (up or down) in debt levels. If the 1%’ers get to harvest the interest payments and the payment of that interest is obtained by higher taxes on the 1%’ers then obviously the overall economic effect is zero (same investor class people gain and “lose” that money). If the debt reduction is gained by increasing taxes on the investor class (at a time when they have more money than there are productive investments), the harm to the economy is minimal. If the debt reduction is obtained by reducing spending that create jobs and benefits to poor people, then it actually seem to do so much harm to the economy that you obtain reduced debt but increased debt/GDP ratio (see southern Europe). Unfortunately the public debate has a hard time talking about a multi-parameter model where the rules adopt/change according to specifics of event.

  8. VennData says:

    Explains the drop in gold.

    Those science-minded-rational Goldbugs and Rick Santelli’s of the world were questioning their fundamental beliefs like a Seven Sister’s first-semester freshman pom squad member.

  9. VennData says:

    I wonder if all those angry white Mitt/Ryan voters who bought this screed can get their money back?

    ​http://www.amazon.com/This-Time-Different-Centuries-Financial/dp/0691152640

  10. MayorQuimby says:

    I would certainly not say it removes *all* of the urgency since the effects of such high debtloads are unknown in the context of an unknown future.

  11. Moss says:

    Has anyone ever used private debt ratios? Or total private and public ratios?

    • Yes. It’s not a pretty picture no matter which ratios you use. Folks can argue about coding errors and sovereign debt thresholds and so on, but I think it’s too late for all that. Fifty years from now the debate about “how much debt can we safely pile onto the GDP” is going to look about as foolish as the medieval theological argument over “how many angels can fit on the head of a pin”.

  12. hue says:

    Reinhart & Rogoff initial response.

  13. Here’s the actual critique paper for those who want to do their own due diligence: http://www.peri.umass.edu/fileadmin/pdf/working_papers/working_papers_301-350/WP322.pdf

    Figures 1-4 tell the story. The “critique” re-slices the data and finds some errors, but the bottom-line conclusion is still the same: higher debt/GDP correlates with lower real growth rates. The critique claims the effect is not as extreme as Reinhart and Rogoff made it look.

    Given that the entire developed world is already carrying too much debt, I think the much more enlightening research would be a comparative study of EXIT STRATEGIES. Austerity, growth, decapitate-and-recapitalize (bad banks to clear bad debts), quantitative easing, selective default, Argentina, Swedish Models etc. But that’s partly because I have a fondness for “Swedish Model” (and I know Barry did too, back in 2008) and I think Iceland deserves a lot more credit for their approach than anyone gives them! Maybe that’s because they don’t buy advertising?

    • flakester says:

      Sounds like it was a mild hit piece, and reactions have been way too excessive.

      • DeDude says:

        You should try to actually read the piece before you just gobble up the narrative given by “Sustainable Gains”. The original claim to fame from R&R’s work, and the way it was used by “austerians”, was that 90% represented some kind of “over-the-cliff-no-way-back” event that we at all cost had to avoid passing. If you open the link and look at figure 4 it does show a correlation, but no indication of a “cliff” at 90% can be found. The closest we get to a “cliff” is the range from 0-40% (but a claim of armageddon as soon as we pass 0% would be a hard sell). When you get past 120% the curve is supported by very few data-points and you can find two cases of growth less than 0.5% as well as two cases of growth over 6%. Are you going to use that data to claim that growth will decrease to 0.5% or increase to 6.0% if we just make sure to get the debt over 120% of GDP? And before you call that suggestion absurd, remember that one of the problems was the selective deletion of data-points by R&R. Anybody presented with the diagram shown in figure 4 with no knowledge (and therefore no agenda) of what the parameters are would say that the data is all over the place and that regardless of statistics these parameters appear to have minimal if any influence on each other (in either direction).

  14. flakester says:

    Reinhart & Rogoff respond (answer: it doesn’t
    change the basic conclusions, debt DOES matter etc. etc.)
    http://www.businessinsider.com/reinhart-and-rogoff-respond-to-critique-2013-4

  15. dirge says:

    The original R&R conclusion was that there was a
    severe breaking point at a 90% debt to GDP ratio. What Herdon et al
    found: “Far from appearing to be a break, average real GDP growth
    in the category of public debt/GDP between 90 and 120 percent is
    2.4 percent, reasonably close to the 3.2 per- cent GDP growth in
    the 60-90 percent category. GDP growth in the new category between
    120 and 150 percent is lower at 1.6 percent but does not fall off a
    nonlinear cliff.” The data was cooked to say what R&R and
    their backers wanted, and it was cooked sloppily. We should
    probably also beware of assigning the causation from high debt
    ratio to low GDP, instead of the reverse. We know quite well what
    happens when GDP decreases and tax revenue goes down, while
    safety-net programs get used more.

    • flakester says:

      From the RR response at BI:

      “We literally just received this draft comment, and will review it in due course.

      On a cursory look, it seems that that Herndon Ash and Pollen also find lower growth when debt is over 90% (they find 0-30 debt/GDP , 4.2% growth; 30-60, 3.1 %; 60-90, 3.2%,; 90-120, 2.4% and over 120, 1.6%). These results are, in fact, of a similar order of magnitude to the detailed country by country results we present in table 1 of the AER paper, and to the median results in Figure 2. And they are similar to estimates in much of the large and growing literature, including our own attached August 2012 Journal of Economic Perspectives paper (joint with Vincent Reinhart) . However, these strong similarities are not what these authors choose to emphasize.

      The 2012 JEP paper largely anticipates and addresses any concerns about aggregation (the main bone of contention here), The JEP paper not only provides individual country averages (as we already featured in Table 1 of the 2010 AER paper) but it goes further and provide episode by episode averages. Not surprisingly, the results are broadly similar to our original 2010 AER table 1 averages and to the median results that also figure prominently.. It is hard to see how one can interpret these tables and individual country results as showing that public debt overhang over 90% is clearly benign.”

      [...]

      “By the way, we are very careful in all our papers to speak of “association” and not “causality” since of course our 2009 book THIS TIME IS DIFFERENT showed that debt explodes in the immediate aftermath of financial crises. This is why we restrict attention to longer debt overhang periods in the JEP paper., though as noted there are only a very limited number of short ones. Moreover, we have generally emphasized the 1% differential median result in all our discussions and subsequent writing, precisely to be understated and cautious , and also in recognition of the results in our core Table 1 (AER paper).

      Lastly, our 2012 JEP paper cites papers from the BIS, IMF and OECD (among others) which virtually all find very similar conclusions to original findings, albeit with slight differences in threshold, and many nuances of alternative interpretation.. These later papers, by they way, use a variety of methodologies for dealing with non-linearity and also for trying to determine causation. Of course much further research is needed as the data we developed and is being used in these studies is new. Nevertheless, the weight of the evidence to date –including this latest comment — seems entirely consistent with our original interpretation of the data in our 2010 AER paper.”

  16. flakester says:

    From the RR response at BI:

    “We literally just received this draft comment, and will review it in due course.

    On a cursory look, it seems that that Herndon Ash and Pollen also find lower growth when debt is over 90% (they find 0-30 debt/GDP , 4.2% growth; 30-60, 3.1 %; 60-90, 3.2%,; 90-120, 2.4% and over 120, 1.6%). These results are, in fact, of a similar order of magnitude to the detailed country by country results we present in table 1 of the AER paper, and to the median results in Figure 2. And they are similar to estimates in much of the large and growing literature, including our own attached August 2012 Journal of Economic Perspectives paper (joint with Vincent Reinhart) . However, these strong similarities are not what these authors choose to emphasize.

    The 2012 JEP paper largely anticipates and addresses any concerns about aggregation (the main bone of contention here), The JEP paper not only provides individual country averages (as we already featured in Table 1 of the 2010 AER paper) but it goes further and provide episode by episode averages. Not surprisingly, the results are broadly similar to our original 2010 AER table 1 averages and to the median results that also figure prominently.. It is hard to see how one can interpret these tables and individual country results as showing that public debt overhang over 90% is clearly benign.”

  17. biotrekker says:

    GDP growth still declines with increased Debt/GDP.

    From Table 3: For Debt/GDP of 90%, the “corrected” results according to the authors refuting Reinhart & Rogoff are 4.2%, 3.1%, 3.2% and 2.2%. Last time I looked, 2.2% growth at >90% Debt/GDP is substantially less than 4.2% at <30% Debt to GDP, or 3.2-3.3% at 30%-90% GDP.

    If the question is whether there is a "break point" at 90% or just a more linear decline, that seems to be a minor point. The conclusions are otherwise the same. More debt/GDP is – on average – associated with lower growth. Kyle Bass suggests that it is even better to look at public debt/gov't "revenue".

  18. Whammer says:

    Imagine for a moment what would happen if a climate scientist released a study that was as seriously flawed as this. I’m thinking that a lot of the folks who are saying “well this doesn’t really change the conclusions” re R&R would be singing a very different tune.

  19. Richard W. Kline says:

    I’m a big fan of Reinhart and Rogoff’s preceding paper on public debt defaults, which was at least in line with prior work on that subject.

    This is an aardvark of another color, however. Anytime researchers decline to publish their data, that is a huge red flag. An arbitrary methodology seems strange, to say the least. As another commenter said, picking a particular debt ratio to analyze can conceal as much as it reveals. If the three criticisms emphasized in the psot are true, R & R simply should never have published this paper period—and they have to know that, which raises questions of motivation. The issue may be simply a rush job to stake out a claim on a particular topic; that happens in academia. One tends to be more suspicious, however . . . .

  20. bear_in_mind says:

    Me thinks RogRein ‘screwed the pooch’ here. Most sentient beings grasp the notion that excessive indebtedness is likely to create drag on growth, if for no other reason than potential demand being siphoned-off by debt servicing. Fine. But RogRein made an informed decision to SELECT the specific debt ratios ranges that they hung their hats on to “prove” their thesis. If that isn’t a form of confirmation bias, I don’t know what is!

  21. Richard W. Kline says:

    Debt and growth in an inverse relationship is fairly well established—but there is no fixed level of ‘how much debt is too much.’ This is the real issue of the paper. Any break point is going to be derived, as countries and their situations vary significantly. The real key is how to treat the exceptions to the trend, i.e. countries which still sustained putatively ‘average’ growth despite relatively high debt. How did they do that? Are they exceptions to a generally bleaker pattern, or is there a reasonable probablity that most or all scenarios _could_ have trended that way to a better outcome? Without a very well-founded methodological approach for treating trend outliers, it is questionable at best whether an effort to derive a ‘common break point’ should even have been attempted as opposed to a case study centered analysis.

    I am doubtful that there is a single growth : debt relationship which applies to all economies, and hence an effort to compare all grow rates is to a significant degree unsound from that perspective. Developing and mature economies have substantially different dynamics, for instance. Economies with fairly sound legal and regulatory environments very likely have different sustainable debt levels than environments which lack them. Debt and growth aren’t the only causally linked activities; for example, what does the payment stream on that debt look like context by context. Amongst the largest conceptual errors in the economic discpiline is an inclination to boil down complex relationships to simple, smooth statistical figures, and this effort has that reek to it.

  22. 873450 says:

    Without passing judgment on Siegel’s motivation, there is a huge difference between a college professor selling a book promoting flawed theory based on his intellectually lazy, unquestioning acceptance of flawed historic data contrast with economists Reinhart-Rogoff selectively including and excluding data to support a flawed conclusion they were paid in advance to hype.

    Sort of like comparing guilty, irresponsible neocons yelling “slam dunk” and blaming faulty CIA intelligence for the Iraq war after it became a disaster contrast with pre-war British intelligence talking points memo reporting neocons were cherry-picking evidence arranged to support their pre-ordained, inevitable decision to invade.

  23. BennyProfane says:

    How much you wanna bet that Ryan and his comrades still use the pre debunked theory as a basis for their ’16 campaign? Like 98% of Americans have even heard of Reinhart-Rogoff.

  24. vavoida says:

    open access, open data ..

    at least the Excel error could have been avoided, if the data in econ papers would be shared

    Open Access to Data: An Ideal Professed but Not Practised

    Out of the sample, 435 researchers (89.14%) neither have a data&code section nor indicate whether and where their data is available. We find that 8.81% of researchers share some of their data whereas only 2.05% fully share.

    http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2224146

  25. DeDude says:

    The model of causation from low GDP (recession) causing more debt is supported by a simple concept proven in pretty much every single recession (has there ever been a recession where tax-receipt did not go down and social expenditures did not go up?). However, the model of debt causing low GDP is based on a flim-flam concept of money being “sucked away” from businesses so they cannot expand and grow the economy. It completely ignores that central banks are the ones that determine availability of money and the rates paid to borrow. If the central banks determine that good solid GDP producing business investments cannot find loans, then they can change their policies to correct that problem – REGARDLESS of how much the national debt is (provided your country has its own currency and central bank).

  26. [...] with the famous Reinhart-Rogoff paper.  (NextNewDeal, Pragmatic Capitalism, Quartz, FT Alphaville, Big Picture, Marginal Revolution, Bonddad Blog, The Atlantic, Modeled [...]

  27. [...] In a new working paper, co-authored with Thomas Herndon, we found that these results were based on a series of data errors and unsupportable statistical [...]