Psychology’s Credibility Crisis

In a recent interview appearing in Discover Magazine, Brian Nosek, Co-founder of the Center for Open Science and speaker at the upcoming BITSS Annual Meeting, discusses the credibility crisis in psychology. 

According to the article, Psychology has lost much of it credibility after a series of published papers were revealed as fraudulent and many other study results were found to be irreproducible.

Fortunately, “psychologists, spurred by a growing crisis of faith, are tackling it [the credibility crisis] head-on. Psychologist Brian Nosek at the University of Virginia is at the forefront of the fight.”  Below are excerpts from Nosek’s interview with Discover Magazine discussing what he and others are doing to increase the rigor of research.

What are you doing about the crisis?

BN: In 2011, colleagues and I launched the Reproducibility Project, in which a team of about 200 scientists are carrying out experiments that were published in three psychology journals in 2008. We want to see how many can reproduce the original result, and what factors affect reproducibility. That will tell us if the problem of false-positive results in the psychology journals is big, small or non-existent…

[W]e built the Open Science Framework (OSF) a web application where collaborating researchers can put all their data and research materials so anyone can easily see them. We also offer incentives by offering “badges” for good practices, like making raw data available. So the more open you are, the more opportunities you have for building your reputation.

What else is being done about it?

BN: At least five psychology journals are inviting people to submit plans for replicating specific studies, which will then get peer-reviewed on the basis of their designs [rather than their results].

The Association for Psychological Science has also launched a project for publishing replications. As a start, 30 teams have signed up to each try to replicate [observations of] a  phenomenon called the “verbal overshadowing effect.”

To learn more about Nosek’s work, read the full interview or come hear him speak at this year’s BITSS Annual Meeting. His talk will start at 2:30 PM (PST) on Thursday, Dec. 11, and will be livestreamed. You can even send questions via Twitter using #bitss2014.

If you’d like to learn how you can incorporate the OSF into your own work, BITSS will be hosting a Collaborative Training (Thursday, Dec. 11, 9.00 AM – 12.00 PM (PST)). During the training, BITSS’ instructors will provide an overview of the OSF and other transparency-enabling tools, including Dataverse and Github.

The full agenda of the conference can be found here. Make sure to register here.