Tom Stanley, friend of BITSS and one of the leaders of the Meta-Analysis of Economics Research Network (MAER-Net), recently published a paper in The Economic Journal with John Ioannidis and Chris Doucouliagos on statistical power in economics research that should be of interest to the BITSS network.
“The power of bias in economics research” is the first large-scale study of the bias, statistical power, and hence the scientific credibility of economics. A survey of 64,076 economic estimates from 159 areas of research and 6,700 empirical studies finds that the median statistical power is 18%, or less. That is, the probability that an empirical economic investigation is able to identify what it seeks is usually 18%, or less. Impotence begets bias. This survey also identifies widespread bias. Typically, reported economic effects are inflated by 100% with one-third inflated by a factor of four or more. In other words, over half of economic research results are reported to be twice as large as they actually are, and one-third are exaggerated to be four times too big. Lastly, 90% of economics findings are under-powered, relative to the widely accepted convention that 80% defines ‘adequate’ power, for half of these areas of economics research.
Download the paper here.