Herndon Responds To Reinhart Rogoff - Business Insider

Ed note: The following is a guest post from University of Massachusetts grad student Thomas Herndon. Herndon shook the economics world last week by debunking an influential paper from Harvard econ professors Carmen Reinhart and Ken Rogoff that claimed countries with debt to GDP above 90% experience much lower growth.

Herndon's paper showed an Excel coding flaw was partly to blame for the result.

After the paper came out, Reinhart and Rogoff admitted the error, but said that their core point remained valid.

In this post, Herndon responds to their response, and says the core point does not remain sound, and that his work does not show that debt-to-GDP above a certain threshold makes a meaningful impact on growth.

I would like to thank Business Insider for inviting me to reflect on the media coverage of the recent paper I co-authored, and to clear up any misconceptions I felt remained.

Overall I think the media coverage has been very fair and supportive. I want to thank the many journalists and media outlets that helped facilitate discussion about this important policy issue, and who have shown a young graduate student, with little media experience, the utmost respect and courtesy. 

I want to address here what I feel are the major misinterpretations of  our work, which will in part require engaging with the claim’s made in Reinhart’s and Rogoff’s response. First, we categorically did not impute any negative motives to the authors; and second, our results are not consistent with and do not confirm their findings. 

I want to start by stating in the strongest possible terms that the purpose of our paper was not to imply that the selective omissions and unconventional weighting were, as R&R asserted in response to us, “intentionally used to bias the results.” The purpose of our paper was strictly to ascertain the veracity of their results.  We know nothing whatsoever about their motives, and did not speculate on this at all in our paper. Throughout our paper we assume that their errors were honest mistakes. We also have honest differences over the appropriate methods for calculating average GDP growth figures.

We did use the terms “selective” and “unconventional” to describe the problems we saw with their paper, and we believe these are accurate characterizations.  “Selective” is an appropriate description because the data were “selected” for exclusion. This does not imply that dishonest motives were at play, only that selection criterion needs to be clearly and transparently presented, so that their work is easily replicable. 

“Unconventional” is appropriate in describing their averaging technique.  To use a baseball analogy, suppose we had a team with two players, and we want to find the team’s overall batting average. The first player has 100 at bats, is successful one-fifth of the time, and therefore has a .200 batting average. The second player has a single at bat, but gets on base in this one at-bat, and so has a perfect 1.000. If we use the Reinhart-Rogoff method, we would equally weight the .200 and 1.000 batting averages, and thus find that the team has an overall .600 batting average. If we used conventional methods of calculating the team’s batting average, it would more or less remain .200. 

The underlying problem is not that their method is necessarily wrong, but that it is particularly sensitive to outliers. This contributed to the “perfect storm” of errors whose combined effect caused the large decline in average GDP. If the only problem was the weighting, this would not have been sufficient to cause a drastic decline in average GDP growth.  However, it was the combination of the weighting system with the exclusion – for whatever reason – that combined to cause the most significant fall in average GDP growth. There is nothing inherently wrong with their weighting system. However it is unusual and it is their obligation to be open and clear in explaining why they used this unusual methodology.

Reinhart and Rogoff make the claim that their unconventional averaging method is easily observable in Table 1, which covers the larger time period 1800-2009, and that their calculations over this longer period are free from error.

Unfortunately, the spreadsheet error actually extends to this table as well, which makes it impossible to ascertain their method from the table. In terms of my own research process, early on in the project I actually asked the very question of whether they had averaged country mean GDP growth rates instead of including each year’s growth experience for each country equally. In a meeting with one of my co-authors, we investigated this very possibility by simply averaging the means provided in the table (in Excel). The computed average GDP growth rate is 2.0, and the median is 2.2, which does not match the average reported by R&R of 1.7 percent, and a median growth figure of 1.9 percent. At the time we thought that this implied that they did not use the country-averaging method.  But it is now clear that the figures they reported resulted from their spreadsheet error. 

Additionally, in Table 1 from the AER publication there appears to be a transcription error. Spain’s average GDP growth in Table 1 is recorded at 2.2 percent. However, in the spreadsheet the published average of 1.7 percent is calculated with Spain’s average GDP growth at 2.8 percent. The correct average and median GDP growth from the spreadsheet are 2.1 percent and 2.3 percent respectively.

Moreover, their spreadsheet error also extends to the entire rest of the summary page of their spreadsheet. Specifically, the error extends to both 1946-2009 and 1800-2009 time periods, mean and median growth, and mean and median inflation. For the entire 1946-2009 sample, rows 30-44 were used, and for the 1800-2009 sample, the rows 5-19 were used. The same countries, Australia, Austria, Belgium, Denmark and Canada, are removed in both samples.  R&R write that Table 1, “does not have the same issues,” but this is not accurate.  Moreover, these computational errors also compromise other calculations on their spreadsheet. 

R&R state that the results of our paper are consistent with, or confirm their  finding—that, even with our criticisms of their work, we nevertheless still  observe economic growth declining significantly when public debt relative to GDP rises above the 90 percent threshold. This is not our interpretation of our work, and indeed the purpose of the second half of our paper is to argue this point. In Table 4 we show that differences in average GDP growth in the categories 30-60 percent, 60-90 percent, and 90-120 percent cannot be statistically distinguished.

In Table 5 we refute their claim that this relationship applies to a wide range of times and places by showing that it weakens as we move forward in time.

Indeed, in the most recent period of 2000-2009, which in almost all cases will be the most relevant set of experiences with respect to current policy debates, average GDP growth when public debt is above 90 percent of GDP is higher than when the public debt/GDP ratio  is between 60 and 90 percent.    The findings in our paper are clearly not consistent with the notion that we consistently observe a sharp fall-off in economic growth when the public debt/GDP ratio exceeds 90 percent.   As for the misconceptions concerning causality, I encourage people to read the contribution by my professor Arin Dube. His treatment of the topic is highly readable and offers strong evidence that causality runs from slow growth to high debt.

There is not one word in our paper which suggests that a high level of government indebtedness is never a problem.  It would be absurd to think that governments never have to worry about their level of indebtedness.  The aim of our paper was much more narrowly focused.  We show that, contrary to R&R, there is no definitive threshold for the public debt/GDP ratio, beyond which countries will invariably suffer a major decline in GDP growth. The implication for policy is that, under particular circumstances, public debt can play a key role in overcoming a recession. The current historical moment, with historically high rates of mass unemployment in both the U.S. and Europe   and with interest rates on U.S. Treasury bonds at historic lows,   is precisely the set of circumstances under which we would expect public borrowing to have large positive effects, with comparably fewer costs. Moreover, it is precisely the set of circumstances under which we expect austerity to have substantial negative effects. 

Click here to read Herndon's full paper >

Now Watch: Everything You Need To Know About Chained CPI, The President's Stealth Cut To Social Security