If you’re anything like us here at the Berkeley Initiative for Transparency in the Social Sciences (BITSS), you’ve been on the lookout for the most notable innovators in research transparency and reproducibility. That’s why we were thrilled to receive 44 nominations for researchers and educators from 18 disciplines and sub-disciplines and 12 countries for the 2016 Leamer-Rosenthal Prizes for Open Social Science. We’re even more excited to announce the nine winners at our fifth Annual Meeting next month! And as fortune would have it, we can give you a sneak peek into the caliber of this year’s nominees—a discussion of the work of our only joint award recipients was featured this week in Nature.
Michèle Nuijten, a doctoral student at Tilburg University, and Sacha Epskamp, an Assisstant Professor at the University of Amsterdam, are the creators of ‘statcheck,’ an R package that allows users to easily and accurately detect p-value flaws in statistical analyses (did we mention it’s free?). These errors, intentional or not, can lead consumers and users of these studies to come to incorrect conclusions such as overestimated or underestimated effect sizes. Such inaccuracies can be especially harmful if they are used to steer policy and decision making.
Ms. Nuijten and Dr. Epskamp, along with other researchers at Tilburg University, took the tool a step further to check studies published in five high-impact psychology journals between 1985 and 2013 and found that nearly half of them contained at least one statistical error. While some of these errors are simple mistakes, many can be seen as symptoms of a publishing system that incentivizes perverse practices such as p-hacking, specification searching, and – in the worst cases – fraud.
“The current scientific system seems to reward individual researchers for quick, flashy, and above all, significant results. For open science to be the new standard, we need to find a way to reward research groups instead of solo scientists, and value methodology above results,” said Ms. Nuijten when asked to discuss the biggest challenges to transparent research. Her hope is that statcheck can “contribute to improving science by preventing misreported statistics from ending up in the literature.”
Statcheck is one of many tools now available for researchers to improve the transparency and reproducibility of their research. And while it is still in beta, it is already being used by journals like Psychological Science to screen for mistakes, as well as by researchers checking their own and others’ work.
The Leamer-Rosenthal Prizes for Open Social Science were named after psychologist Robert Rosenthal (UC Riverside) and economist Ed Leamer (UCLA) – two pioneers of the research transparency movement. The prizes were created to reward those driving change in social science research by educating others, developing tools to facilitate openness, and carrying out transparent and reproducible science themselves.
Though public confidence in the integrity of scientific research seems to have waivered in recent years, a majority of the American public trusts scientists to act in their interest. And if four years of working here at BITSS has shown us anything, it’s that the research community is chock-full of people working hard, not only to do good science themselves, but also to make it easier for others to do the same.
This year’s prize recipients will join ten exceptional educators and researchers awarded the prize last year. Among those winners are J. Scott Long, who authored multiple textbooks on reproducibility, Sean Grant, who helped develop the CONSORT-SPI standards for randomized control trial reporting, and Eva Vivalt, who created AidGrade to share meta-analysis results from international development program evaluations. Also among the 2015 winners are David Broockman, Joshua Kalla, and Peter Aronow, who created quite a stir last year when their attempt to reproduce a high-profile study on the ability of canvassers to change American voters’ minds about same-sex marriage led to its retraction from Science magazine.
Ms. Nuijten and Dr. Epskamp will receive their prizes, along with seven other researchers and educators, at the 2016 BITSS Annual Meeting on December 15. They will also participate in a panel to discuss their perspectives on the state and future of transparency in social science, as well as their roles in the movement.
The Annual Meeting is free to attend and will be held at the David Brower Center in Berkeley, California this December 15-16, 2016. A detailed agenda and registration link can be found here. The meeting will also be live streamed on the BITSS website here.
 Carey Funk and Brian Kennedy, “The Politics of Climate,” Pew Research Center, October 4, 2016, http://www.pewinternet.org/2016/10/04/the-politics-of-climate/.