The Power of Bias in Economics Research

Tom Stanley, friend of BITSS and one of the leaders of the Meta-Analysis of Economics Research Network (MAER-Net), recently published a paper in The Economic Journal with John Ioannidis and Chris Doucouliagos on statistical power in economics research that should be of interest to the BITSS network.

From Tom:

“The power of bias in economics research” is the first large-scale study of the bias, statistical power, and hence the scientific credibility of economics.  A survey of 64,076 economic estimates from 159 areas of research and 6,700 empirical studies finds that the median statistical power is 18%, or less.  That is, the probability that an empirical economic investigation is able to identify what it seeks is usually 18%, or less.  Impotence begets bias. This survey also identifies widespread bias. Typically, reported economic effects are inflated by 100% with one-third inflated by a factor of four or more.  In other words, over half of economic research results are reported to be twice as large as they actually are, and one-third are exaggerated to be four times too big.  Lastly, 90% of economics findings are under-powered, relative to the widely accepted convention that 80% defines ‘adequate’ power, for half of these areas of economics research.

Download the paper here.

Leave a Reply