Reproducibility Project: Psychology Released in Science

Garret Christensen–BITSS Project Scientist

A paper I’ve been excited about for a while now has just been published in Science: “Estimating the Reproducibility of Psychological Science” by the Open Science Collaboration. The paper is the work of the Reproducibility Project: Psychology, coordinated by the Center for Open Science (COS) and funded by the Laura and John Arnold Foundation (a funder of BITSS). Essentially, 270 researchers repeated 100 psychology experiments, and found that fewer than half of the replications gave the same results as the original.

From COS’ press release:

Fewer than half of the original findings were successfully replicated. This held true across multiple different criteria of success. The team noted three basic reasons this might occur: 1) Even though most replication teams worked with the original authors to use the same materials and methods, small differences in when, where, or how the replication was carried out might have influenced the results. 2) The replication might have failed to detect the original result by chance. 3) The original result might have been a false positive. Johanna Cohoon, another Project Coordinator from the Center for Open Science, concluded that, “The findings demonstrate that reproducing original results may be more difficult than is presently assumed, and interventions may be needed to improve reproducibility.” In keeping with the goals of openness and reproducibility, every replication project posted its methods on a public website, and later added their raw data and computer code for reproducing their analyses.

See the full paper in Science.

Follow the news coverage via Google News.

Where do we go from here? Join the conversation on twitter with #ReproducibilityProject.