Research Library
We invest in innovative meta-scientific research, as well as in the development and testing of new tools and methods to contribute to a growing body of evidence on problems and solutions in science. Our research portfolio includes work led by BITSS investigators, as well as by our partners and grantees of our Social Science Meta-Analysis and Research Transparency (SSMART) grants.
This Research Library catalogs all research projects supported or led by BITSS. All research projects are categorized by i) program, ii) metascientific topic, and iii) discipline. Filter results by applying criteria along these parameters.
Find an overview of our research portfolio here.
The evidence-based policy movement promotes the use of empirical evidence to inform policy decision-making. While this movement has gained traction over the last two decades, several concerns about the credibility of empirical research have been identified in scientific disciplines that... Read More
Fernando Hoces Sean Grant Edward Miguel
Scott Desposato and George Avelino conducted the first meta-analysis and reproducibility analysis of political science in Brazil using all articles published in the last five years in the three leading Brazilian political science and general social science journals, including the... Read More
George Avelino Scott Desposato
The evidence-based community has championed the public registration of pre-analysis plans (PAPs) as a solution to the problem of research credibility, but without any evidence that PAPs actually bolster the credibility of research. Ofosu and Posner analyze a representative sample... Read More
George Ofosu Daniel Posner
Aggregating data across studies is a central challenge in ensuring cumulative, reproducible science. Meta-analysis is a key statistical tool for this purpose. The goal of this project is to support meta-analysis using MetaLab, an interface and central repository. MetaLab supports... Read More
Christina Bergmann Sho Tsuji Molly Lewis Mika Braginsky Page Piccinini Alejandrina Cristia Michael C. Frank
The introduction of confidence at 95 percent or 90 percent has led the academic community to accept more easily starry stories with marginally significant coefficients than starless ones with insignificant coefficients. In February 2015, the editors of eight health economics... Read More
Cristina Blanco-Perez Abel Brodeur
“Transparent and Reproducible Social Science Research: How to do open science” is the first textbook to offer a comprehensive introduction to open science tools and methods. Use code 17M6662 at checkout to take advantage of a 30% discount in purchases... Read More
Garret Christensen Jeremy Freese Edward Miguel
This study estimates the effect of data sharing on the citations of academic articles, using journal policies as a natural experiment. We begin by examining 17 high-impact journals that have adopted the requirement that data from published articles be publicly... Read More
Garret Christensen Allan Dafoe Edward Miguel Don Moore Andrew K. Rose
There are a growing number of studies using mediation analysis to understand the mechanisms of health interventions and exposures. Recent work has shown that the reporting of these studies is heterogenous and incomplete. This problem stifles clinical application, reproducibility, and... Read More
Hopin Lee James H. McAuley Steven Kamper Nicolas Henschke Christopher M. Williams
Imai, Rutter, and Camerer examine 220 estimates of the present-bias parameter from 28 articles using the Convex Time Budget protocol. The literature shows that people are on average present biased, but the estimates exhibit substantial heterogeneity across studies. There is... Read More
Colin Camerer Taisuke Imai
Eric-Jan Wagenmakers and colleagues developed a suite of meta-analytic techniques for Bayesian evidence synthesis, addressing a series of challenges that currently constrain classical meta-analytic procedures. These techniques include (1) an application of bridge sampling to obtain Bayes factors for random-effects... Read More
E.J. Wagenmakers Raoul Grasman Quentin F. Gronau Felix Schönbrodt
Lenz and Sahn examine how often research findings depend on suppression effects, or covariate-induced increases in effect sizes. Researchers generally scrutinize suppression effects as they want reassurance that researchers have a strong explanation for effect size increases, especially when the... Read More
Gabriel Lenz Alexander Sahn
Cleo Protogerou, Martin Hagger, and Blair Johnson developed and tested an integrated theory of the determinants of condom use among young people in Sub-Saharan Africa using meta-analytic path analysis. The meta-analysis encompassed fifty-five studies (N = 55,069), comprising 72 independent... Read More
Cleo Protogerou Martin Hagger Blair T. Johnson
This project aims to replicate and combine three recent experiments on capital transfers to poor households in two distinct phases. The first phase will produce three concise internal replications. These will be accompanied by a report detailing the challenges faced... Read More
Elliott Collins Ethan Ligon Reajul Chowdhury
In this study, Joshua Polanin and Mary Terzian provide an explanation of why primary study authors are unwilling to share their data and evaluate whether sending a data-sharing agreement affects participants’ willingness to share IPD. They sampled and surveyed more... Read More
Joshua Polanin Mary Terzian
Pre-registration is a straightforward way to make science more transparent, and control Type 1 error rates. Pre-registration is often presented as beneficial for science in general, but rarely as a practice that leads to immediate individual benefits for researchers. One... Read More
Daniel Lakens
There is growing interest in research transparency and reproducibility in economics and other fields. We survey existing work on these topics within economics and discuss the evidence suggesting that publication bias, inability to replicate and specification searching remain widespread problems... Read More
Edward Miguel Garret Christensen
Protocols improve the reproducibility and accessibility of social science research. Given deficiencies in trial protocol quality, the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) Statement provides an evidence-based set of items to describe in protocols of clinical trials on... Read More
Sean Grant
Scientists and consumers of scientific knowledge can struggle to synthesize the quickly evolving state of empirical research. Even where recent, comprehensive literature reviews and meta-analyses exist, there is frequent disagreement on the criteria for inclusion and the most useful partitions... Read More
Solomon Hsiang James Rising
Burlig, Preonas, and Woerman develop a power calculation technique and accompanying software package for experiments that use panel data. They generalize Frison and Pocock (1992) to fully arbitrary error structures, thereby extending McKenzie (2012) to allow for non-constant serial correlation.... Read More
Fiona Burlig Matt Woerman Louis Preonas
Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data... Read More
Daniel Lakens Marcel van Assen Farid Anvari Katherine Corker James Grange Heike Gerger Fred Hasselman Jacklyn Koyama Cosima Locher Ian Miller Elizabeth Page-Gould Felix Schönbrodt Amanda Sharples Barbara Spellman Shelly Zhou
This paper develops methods to aggregate evidence on distributional treatment effects from multiple studies conducted in different settings and applies them to the microcredit literature. Several randomized trials of expanding access to microcredit found substantial effects on the tails of... Read More
Rachael Meager
The proposed project will utilize p-curve, a new meta-analytic tool to assess the evidentiary value of studies from social psychology and behavioral marketing. P-curves differs from meta-analytic methods by analyzing the distribution of p-values to determine the likelihood that a... Read More
Leif Nelson
As methods for internal validity improve, methodological concerns have shifted toward assessing how well the research community can extrapolate from individual studies. Under recent federal granting initiatives, over $1 billion has been awarded to education programs that have been validated... Read More
Sean Tanner
Publication bias is a substantial problem for the credibility of research in general and of meta-analyses in particular, as it yields overestimated effects and may suggest the existence of non-existing effects. Although there is consensus that publication bias exists, how... Read More
Robbie van Aert Jelte Wicherts Marcel van Assen
Replication is essential for building confidence in research studies, yet it is still the exception rather than the rule. That is not necessarily because funding is unavailable — it is because the current system makes original authors and replicators antagonists.... Read More
Paul Gertler
Under-powered studies combined with low prior beliefs about intervention effects increase the chances that a positive result is overstated. We collect prior beliefs about intervention impacts from 125 experts to estimate the false positive and false negative report probabilities (FPRP... Read More
Eva Vivalt Aidan Coville
Publication bias leads consumers of research to observe a selected sample of statistical estimates calculated by producers of research. We calculate critical values for statistical significance that undo the distortions created by this selection effect, assuming that the only source... Read More
Justin McCrary Garret Christensen Daniele Fanelli
In a standard scientific analysis, one analyst or team presents a single analysis of a data set. However, there are often a variety of defensible analytic strategies that could be used on the same data. Variation in those strategies could... Read More
Garret Christensen
The Transparency and Openness Promotion (TOP) Committee met at the Center for Open Science in Charlottesville, Virginia, in November 2014 to address one important element of the incentive systems: journals' procedures and policies for publication. The committee consisted of disciplinary... Read More
Brian Nosek
There is a growing appreciation for the advantages of experimentation in the social sciences. Policy-relevant claims that in the past were backed by theoretical arguments and inconclusive correlations are now being investigated using more credible methods. Changes have been particularly... Read More
Edward Miguel Colin Camerer Kate Casey J. Cohen Kevin Esterling Alan Gerber Rachel Glennerster Donald P. Green Macartan Humphreys Guido Imbens Temina Madon Leif Nelson Brian Nosek Maya Petersen Richard Sedlmayr Joseph Simmons Mark van der Laan
Page 1 of 1