Transcript Examining the Replication Crisis

Examining the Replication
Crisis
Thomas Herndon, Ph.D Candidate
Department of Economics and Political Economy Research Institute
University of Massachusetts Amherst
4-8-2016
Herndon, Ash, Pollin (2014). Cambridge Journal of Economics
Replication in Economics
• Chang and Li (2015) – “Because we are able to replicate less than half of the papers in our sample
even with help from the authors, we assert that economics research is usually not replicable.”
• Hammermesh (2007) – “Our expressions of preference are cheap talk. The profession provides
few incentives for most active economists to produce replications of others’ research, and
similarly few to increase the believability of one’s own research by testing ideas on multiple sets
of data…Editors need to take the lead by providing sufficient incentives for top-flight authors of
empirical work to engage in pure and scientific replication, and by insisting on within-study
scientific replication in many more articles. Without these changes occasional paeans to the
virtues of replication are as likely to enhance the scientific soundness of empirical research in
economics as programmes that urge abstinence are to reduce teenagers’ sexual activity.”
• McCullough, McGeary, and Harrison (2008) – “All the long-standing archives at economics
journals do not facilitate the reproduction of published results. The data-only archives at Journal
of Business and Economic Statistics and Economic Journal fail in part because most authors do not
contribute data. Results published in the FRB St. Louis Review can rarely be reproduced using the
data+code in the journal archive.”
• Hoffler (2013) – “After exploring several dozen studies published in highly ranked journals we
have not yet determined a single case where we see replicability is fully ensured.” Proposes to
better incorporate replication in graduate training, and provide online outlets.