Statistical Significance Testing in Economics

The origins of testing scientific models with statistical techniques go back to 18th-century mathematics. However, the modern theory of statistical testing was primarily developed through the work of Sir R.A. Fisher, Jerzy Neyman, and Egon Pearson in the interwar period. Some of Fisher’s papers on testing were published in economics journals (Fisher, 1923, 1935) and exerted a notable influence on the discipline. The development of econometrics and the rise of quantitative economic models in the mid-20th century made statistical significance testing a commonplace, albeit controversial, tool within economics.

  In the debate about significance testing, methodological controversies intertwine with epistemological issues and sociological developments. Our aim in this chapter is to explain these connections and to show how the use of, and the debate about, significance testing in economics differs from other social sciences, such as psychology.

  Section 2 explains the basic principles of statistical significance testing with particular attention to its use in economics. In Section 3, we sketch how significance testing became entrenched in economics, and we highlight some early criticisms. Section 4 deals with Ziliak and McCloskey’s criticism that significance tests, and the economists who apply them, confuse statistical and proper economic significance (i.e., effect size). Section 5 relates significance testing to the problem of publication bias in science and compares the debates about significance testing in economics and in psychology. Section 6 wraps up and briefly discusses some suggestions for methodological improvement.