Research Library

We invest in innovative meta-scientific research, as well as in the development and testing of new tools and methods to contribute to a growing body of evidence on problems and solutions in science. Our research portfolio includes work led by BITSS investigators, as well as by our partners and grantees of our Social Science Meta-Analysis and Research Transparency (SSMART) grants.

This Research Library catalogs all research projects supported or led by BITSS. All research projects are categorized by i) program, ii) metascientific topic, and iii) discipline. Filter results by applying criteria along these parameters.


  • Discipline

30 Results

The evidence-based policy movement promotes the use of empirical evidence to inform policy decision-making. While this movement has gained traction over the last two decades, several concerns about the credibility of empirical research have been identified in scientific disciplines that... Read More

BITSS ScholarsInterdisciplinaryMethods and ToolsPublic Policy
Fernando Hoces Sean Grant Edward Miguel

Scott Desposato and George Avelino conducted the first meta-analysis and reproducibility analysis of political science in Brazil using all articles published in the last five years in the three leading Brazilian political science and general social science journals, including the... Read More

Evaluations of Transparency and ReproducibilityMeta-Analyses and Systematic ReviewsPolitical ScienceSSMART
George Avelino Scott Desposato

The evidence-based community has championed the public registration of pre-analysis plans (PAPs) as a solution to the problem of research credibility, but without any evidence that PAPs actually bolster the credibility of research. Ofosu and Posner analyze a representative sample... Read More

EconomicsEvaluations of Transparency and ReproducibilityInterdisciplinaryMethods and ToolsPolitical ScienceSSMART
George Ofosu Daniel Posner

Aggregating data across studies is a central challenge in ensuring cumulative, reproducible science. Meta-analysis is a key statistical tool for this purpose. The goal of this project is to support meta-analysis using MetaLab, an interface and central repository. MetaLab supports... Read More

Cognitive ScienceInterdisciplinaryMeta-Analyses and Systematic ReviewsPsychologySSMART
Christina Bergmann Sho Tsuji Molly Lewis Mika Braginsky Page Piccinini Alejandrina Cristia Michael C. Frank

The introduction of confidence at 95 percent or 90 percent has led the academic community to accept more easily starry stories with marginally significant coefficients than starless ones with insignificant coefficients. In February 2015, the editors of eight health economics... Read More

EconomicsEvaluations of Transparency and ReproducibilitySSMART
Cristina Blanco-Perez Abel Brodeur

“Transparent and Reproducible Social Science Research: How to do open science” is the first textbook to offer a comprehensive introduction to open science tools and methods. Use code 17M6662 at checkout to take advantage of a 30% discount in purchases... Read More

BITSS ScholarsEconomicsEvaluations of Transparency and ReproducibilityMethods and ToolsPolitical SciencePsychologyScholarly Communication and Peer Review
Garret Christensen Jeremy Freese Edward Miguel

This study estimates the effect of data sharing on the citations of academic articles, using journal policies as a natural experiment. We begin by examining 17 high-impact journals that have adopted the requirement that data from published articles be publicly... Read More

BITSS ScholarsEconomicsPolitical ScienceScholarly Communication and Peer Review
Garret Christensen Allan Dafoe Edward Miguel Don Moore Andrew K. Rose

There are a growing number of studies using mediation analysis to understand the mechanisms of health interventions and exposures. Recent work has shown that the reporting of these studies is heterogenous and incomplete. This problem stifles clinical application, reproducibility, and... Read More

Methods and ToolsPublic HealthSSMART
Hopin Lee James H. McAuley Steven Kamper Nicolas Henschke Christopher M. Williams

Eric-Jan Wagenmakers and colleagues developed a suite of meta-analytic techniques for Bayesian evidence synthesis, addressing a series of challenges that currently constrain classical meta-analytic procedures. These techniques include (1) an application of bridge sampling to obtain Bayes factors for random-effects... Read More

Methods and ToolsPsychologySSMART
E.J. Wagenmakers Raoul Grasman Quentin F. Gronau Felix Schönbrodt

How often do articles depend on suppression effects for their findings? How often do they disclose this fact? By suppression effects, we mean covariate-induced increases in effect sizes. Researchers generally scrutinize suppression effects as they want reassurance that researchers have... Read More

Evaluations of Transparency and ReproducibilityMeta-Analyses and Systematic ReviewsPolitical ScienceSSMART
Gabriel Lenz Alexander Sahn

This project aims to replicate and combine three recent experiments on capital transfers to poor households in two distinct phases. The first phase will produce three concise internal replications. These will be accompanied by a report detailing the challenges faced... Read More

EconomicsEvaluations of Transparency and ReproducibilityMethods and ToolsSSMART
Elliott Collins Ethan Ligon Reajul Chowdhury

Sharing individual participant data (IPD) among researchers, upon request, is an ethical and responsible practice. Despite numerous calls for this practice to be normalized, primary study authors are often unwilling to share IPD, even for use in meta-analyses. In this... Read More

Methods and ToolsPsychologyPublic HealthPublic PolicyScholarly Communication and Peer ReviewSSMART
Joshua Polanin Mary Terzian

There is growing interest in research transparency and reproducibility in economics and other fields. We survey existing work on these topics within economics and discuss the evidence suggesting that publication bias, inability to replicate and specification searching remain widespread problems... Read More

BITSS ScholarsEconomicsEvaluations of Transparency and ReproducibilityMethods and Tools
Edward Miguel Garret Christensen

Protocols improve reproducibility and accessibility of social science research. Given deficiencies in trial protocol quality, the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) Statement provides an evidence-based set of items to describe in protocols of clinical trials on biomedical interventions. This manuscript... Read More

Methods and ToolsSocial ScienceSSMART
Sean Grant

How should researchers design experiments to detect treatment effects with panel data? In this paper, we derive analytical expressions for the variance of panel estimators under non-i.i.d. error structures, which inform power calculations in panel data settings. Using Monte Carlo... Read More

EconomicsMethods and ToolsSSMART
Fiona Burlig Matt Woerman Louis Preonas

Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data... Read More

Evaluations of Transparency and ReproducibilityMeta-Analyses and Systematic ReviewsPsychologySSMART
Daniel Lakens Marcel van Assen Farid Anvari Katherine Corker James Grange Heike Gerger Fred Hasselman Jacklyn Koyama Cosima Locher Ian Miller Elizabeth Page-Gould Felix Schönbrodt Amanda Sharples Barbara Spellman Shelly Zhou

The proposed project will utilize p-curve, a new meta-analytic tool to assess the evidentiary value of studies from social psychology and behavioral marketing. P-curves differs from meta-analytic methods by analyzing the distribution of p-values to determine the likelihood that a... Read More

Evaluations of Transparency and ReproducibilityPsychologySSMART
Leif Nelson

As methods for internal validity improve, methodological concerns have shifted toward assessing how well the research community can extrapolate from individual studies. Under recent federal granting initiatives, over $1 billion has been awarded to education programs that have been validated... Read More

EconomicsEducationEvaluations of Transparency and ReproducibilityPublic PolicySSMART
Sean Tanner

Publication bias is a substantial problem for the credibility of research in general and of meta-analyses in particular, as it yields overestimated effects and may suggest the existence of non-existing effects. Although there is consensus that publication bias exists, how... Read More

Evaluations of Transparency and ReproducibilityLife SciencesMedicineMeta-Analyses and Systematic ReviewsMetascience (Methods and Archival Science)PsychologySSMART
Robbie van Aert Jelte Wicherts Marcel van Assen

Under-powered studies combined with low prior beliefs about intervention effects increase the chances that a positive result is overstated. We collect prior beliefs about intervention impacts from 125 experts to estimate the false positive and false negative report probabilities (FPRP... Read More

EconomicsEvaluations of Transparency and ReproducibilitySSMART
Eva Vivalt Aidan Coville

Publication bias leads consumers of research to observe a selected sample of statistical estimates calculated by producers of research. We calculate critical values for statistical significance that undo the distortions created by this selection effect, assuming that the only source... Read More

BITSS ScholarsMetascience (Methods and Archival Science)Methods and ToolsSocial Science
Justin McCrary Garret Christensen Daniele Fanelli

The Transparency and Openness Promotion (TOP) Committee met at the Center for Open Science in Charlottesville, Virginia, in November 2014 to address one important element of the incentive systems: journals' procedures and policies for publication. The committee consisted of disciplinary... Read More

BITSS ScholarsInterdisciplinaryMethods and ToolsScholarly Communication and Peer Review
Brian Nosek

There is a growing appreciation for the advantages of experimentation in the social sciences. Policy-relevant claims that in the past were backed by theoretical arguments and inconclusive correlations are now being investigated using more credible methods. Changes have been particularly... Read More

BITSS ScholarsInterdisciplinaryMethods and ToolsScholarly Communication and Peer ReviewSocial Science
Edward Miguel Colin Camerer Kate Casey J. Cohen Kevin Esterling Alan Gerber Rachel Glennerster Donald P. Green Macartan Humphreys Guido Imbens Temina Madon Leif Nelson Brian Nosek Maya Petersen Richard Sedlmayr Joseph Simmons Mark van der Laan

Page 1 of 1