A recent article from The Economist featuring John Ioannidis’ Meta-Research Innovation Center (METRICS), whose work to advance the credibility of research will be presented next week at the BITSS Annual Meeting.
“Why most published research findings are false” is not, as the title of an academic paper, likely to win friends in the ivory tower. But it has certainly influenced people (including journalists at The Economist). The paper it introduced was published in 2005 by John Ioannidis, an epidemiologist who was then at the University of Ioannina, in Greece, and is now at Stanford. It exposed the ways, most notably the overinterpreting of statistical significance in studies with small sample sizes, that scientific findings can end up being irreproducible—or, as a layman might put it, wrong […] Dr Ioannidis has been waging war on sloppy science ever since, helping to develop a discipline called meta-research (ie, research about research).
METRICS’ mission is to “identify and minimize persistent threats to medical-research quality.” These include irreproducibility of research findings (the inability of external researchers to reproduce someone else’s work, most often because research data is not shared or data manipulations are not correctly detailed), funding inefficiencies (supporting flawed research), and publication bias (not all studies that are conducted get published, and the ones which do tend to be show significant results, leaving a skewed impression of the evidence).
The fields of medicine and public health have done tremendous progress addressing these issues. Some social science disciplines are catching up, as Psychology’s Many Labs Replication Project testifies. But the norms and incentives that govern the social sciences make certain challenges specific to its disciplines. What can social science learn from medicine’s reproducibility crisis? That’s the main question to be addressed by John Ioannidis next week at the BITSS Annual Meeting. You can register here.