BITSS Affiliates Advocate for Higher Transparency Standards in Science Magazine

In the January 3, 2014 edition of Science Magazine, an interdisciplinary group of 19 BITSS affiliates reviews recent efforts to promote transparency in the social sciences and make the case for more stringent norms and practices to help boost the quality and credibility of research findings.

The authors, led by UC Berkeley economist Ted Miguel, deplore a dysfunctional reward structure in which statistically significant, novel, and theoretically tidy results get published more easily that null, complicated, or replication outcomes. This misalignment between scholarly incentives and scholarly values, the authors argue, spur researchers to present their data in a way that is more “publishable” – at the expense of accuracy.

Coupled with limited accountability for researchers’ errors and mistakes, this problem has had the effect of producing a somewhat distorted body of evidence that exaggerate the effectiveness of social and economic programs. The stakes are high because policy decisions based on flawed research affect millions of people’s lives.

The Cover of the January 3, 2014 Edition of Science
The Cover of the January 3, 2014 Edition of Science

To address this issue, the interdisciplinary group suggests the adoption of three emerging set of practices by the scientific community:

  1. Systematic disclosure of data collection and analysis methods (i.e. transparency in design);
  2. Registration of pre-analysis plans in which researchers lay out how they will analyze their data before they have actually collected the data (i.e. transparency in intentions);
  3. Open data and research materials to allow others to test and extend reported results (i.e. transparency in analysis).

The Open Science Framework (OSF), the American Economic Association’s (AEA) registry for randomized controlled trials or the Experiments in Governance and Politics’ (EGAP) registry are some new tools that now make it easy for researchers to archive and share study designs and data.

The authors recognize that the adoption of these new practices by the social science community must be gradual – being implemented in a way that does not create excess burden or stifle creativity – but assert that the benefits from increased transparency will have a huge impact on scientific progress and strengthen the quality and credibility of the scientific evidence on which social policies are based.

The Science article was featured by UC Berkeley News Center and in multiple blogs across the web (Phys.org, Science 2.0, Science Codex, Materia, El Observador, etc.).