Open Science Success Stories

The Open Research Funders Group curates the Open Science Success Stories, a database of examples of how openness has benefited researchers and broader society.

Data Citations module

Created by the Federal Reserve Bank of St. Louis, this module introduces students to the key elements of data citations. See also related modules for Data Literacy.

Handbook on Using Administrative Data for Research and Evidence-Based Policy

Co-edited by Shawn Cole, Iqbal Dhaliwal, Anja Sautmann, and Lars Vilhuber and published by J-PAL’s Innovations in Data and Experiments for Action Initiative (IDEA), this handbook includes case studies of large-scale randomized evaluations using private and national government administrative data, and technical guidance to support partnerships with governments, nonprofits, or firms to…

Survey of Registered Reports Editors

Between December 15, 2017 and January 31, 2018, BITSS surveyed the editors of 76 academic journals which at the time, accepted submissions in the Registered Report (RR) format. Find summary statistics of the results in this document.

CRediT (Contributor Roles Taxonomy)

CRediT (Contributor Roles Taxonomy) is high-level taxonomy, including 14 roles, that can be used to represent the roles typically played by contributors to scientific scholarly output. The roles describe each contributor’s specific contribution to the scholarly output.

Comparison of multiple hypothesis testing commands in Stata

In this post on the Development Impact blog, David McKenzie (World Bank) compares various Stata packages used for multiple hypothesis testing adjustments and discusses settings where each package is best applied.

Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Educational Expansion

Created by Catalyst Melissa Sharp, this is an open-source repository for epidemiological research methods and reporting skills for observational studies, structured based on the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement. Use it to discover new methods and reporting guidelines and contribute through the GitHub repository (https://github.com/sharpmel/STROBECourse/).

Pre-Analysis Plans for Observational Research

In her presentation at RT2 DC in 2019, Fiona Burlig (University of Chicago) provides advice on how one can credibly pre-register an observational research project. Also see Burlig’s 2018 paper that describes three scenarios for pre-registration of observational work, including i) cases where researchers collect their own data; ii) prospective studies; and…

Data for Development Impact (Resource Guide)

“Data for Development Impact: The DIME Analytics Resource Guide” is intended to serve as an introduction to the primary tasks required in development research, from experimental design to data collection to data analysis to publication. It serves as a companion to the DIME Wiki and is produced by DIME Analytics.

Open Science Module for Behavioral Science graduate course

Instructors Kelly Zhang (MIT GOV/LAB) and Chaning Jang (Busara) integrated a module on research transparency and the use of pre-analysis plans as part of the Behavioral Science in the Field course designed for graduate students who use behavioral science games as part of their research.

J-PAL Guide to De-Identifying Data

Developed by J-PAL’s Sarah Kooper, Anja Sautmann, and James Turrito, this guide includes: An overview of personally identifiable information (PII) and the responsibility of data users not to use data to try to identify human subjects Recommendations for handling direct identifiers (such as full name, social security number, or phone number), as…

J-PAL Guide to Publishing Research Data

Developed by J-PAL’s Sarah Kooper, Anja Sautmann, and James Turrito, this guide includes: A list of considerations to make before publishing data, such as what information was provided to study participants and the IRB, the sensitivity of the data collected, and legal requirements Sample consent form language that will allow future publication…

Data Sharing Checklist for NGOs and Practitioners

This checklist developed by Teamscope can help NGOs and Practitioners understand the common pitfalls in open data, and how open data impacts every step of a project’s pipeline, from proposal writing to dissemination.

Videos: Research Transparency and Reproducibility Training (RT2) – Washington, D.C.

BITSS hosted a Research Transparency and Reproducibility Training (RT2) in Washington DC, September 11-13, 2019. This was the eighth training event of this kind organized by BITSS since 2014. RT2 provides participants with an overview of tools and best practices for transparent and reproducible social science research. Click here to videos of…

Replicability Seminar

Course syllabus for “Replicability Seminar”, an advanced undergraduate and graduate-level course led by Simine Vazire.

Open Data Metrics: Lighting the Fire

In this book, Daniella Lowenberg and colleagues describe the journey towards open data metrics, prompting community discussion and providing implementation examples along the way. Data metrics are a pre-condition to realize the benefits of open data sharing practices.

BITSS Registered Reports Literature Review

Prepared by BITSS, this literature review includes information on the distinguishing features and advantages of registered reports, as well as challenges involved in its implementation.

Nextjournal

Nextjournal is a container tool with features like polyglot notebooks, automatic versioning and real-time collaboration.

Meta

Meta is an interactive search tool that indexes research in the biomedical sciences. Users can create personalized news feeds by selecting from a range of concepts, journals, preprints, or papers of interest.

Frontiers in Pre-Registration in Economics – Ted Miguel

This presentation by Ted Miguel was given at the Transparency, Reproducibility and Credibility Research Symposium at the World Bank on 9/10/2019. You can find videos of other talks from the Symposium in this playlist.

Transparent and Open Social Science Research (FR)

Demand is growing for evidence-based policy making, but there is also growing recognition in the social science community that limited transparency and openness in research have contributed to widespread problems. With this course created and administered by BITSS, you can explore the causes of limited transparency in social science research, as well…

Software Carpentry

Software Carpentry offers online tutorials for data analysis including Version Control with Git, Using Databases and SQL, Programming with Python, Programming with R and Programming with MATLAB.

BITSS training survey templates

BITSS developed templates for pre- and post-training surveys that can be used by instructors to record learning outcomes in research transparency and reproducibility training events. The links below enable access as an editor; please make a copy of each form to use it for your own purposes:

Data Carpentry Lessons

Developed by Data Carpentry, these lessons can be used across the social sciences to teach data cleaning, management, analysis, and visualization. R is the base language for instruction, and there are no pre-requisites in terms of prior knowledge about this topic.

Observational PAP Guide

In her preprint titled “Improving transparency in observational social science research: A pre-analysis plan approach”, Fiona Burlig (University of Chicago) presents three scenarios in which study preregistration and pre-analysis plans (PAPs) can be credibly applied in non-experimental settings: cases where researchers collect their own data; prospective studies; and research using restricted-access data.…

ResonsibleData.io

Using data for social change work offers many opportunities, but it brings challenges, too. The RD community develops practical ways to deal with the unintended consequences of using data in social change work, establishes best practices, and shares approaches between leading thinkers and doers from different sectors. We discuss thorny topics in-person,…

Registry for International Development Impact Evaluations (RIDIE)

Administered by the International Initiative for Impact Evaluation (3ie), the Registry for International Development Impact Evaluations (RIDIE) is a registry of impact evaluations related to development in low and middle income countries. RIDIE will register any development impact evaluation that rigorously attempts to estimate the causal impacts of a program, including but…

Catalog of open source licenses

Using this online tool, you can choose an open source license to clearly articulate the conditions under which others can use, distribute, modify or contribute to your software and non-software projects.

ARDC FAIR Data self-assessment tool

This checklist, developed by the Australian Research Data Commons (ARDC) may help researchers make their datasets FAIRer: findable, accessible, interoperable and re-usable.

BITSS Pre- and post-training survey templates

BITSS developed templates for pre- and post-training surveys that can be used by instructors to record learning outcomes in research transparency and reproducibility training events. The links below enable access as an editor; please make a copy of each form to use it for your own purposes: Pre-training survey Post-training survey

Conda

Conda is an open source package management system and environment management system that runs on Windows, macOS and Linux. Conda installs, runs and updates packages and their dependencies and is operable in multiple languages, including Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN.

Stage 1 Registered Report Submission Template

BITSS prepared a template to assist authors in the preparation of their Stage 1 Proposal submissions to the Journal of Development Economics. The template expands on features that are commonly reported in pre-analysis plans in development economics, and includes a checklist to help authors record different parts of the research design.

Whole Tale

Whole Tale is an infrastructure that allows users to share data, methods and analysis protocols, and final research outputs in a single, executable object (“living publication” or “tale”) alongside any research publication. Learn more here.

NRIN Collection of Resources on Research Integrity

Curated by the Netherlands Research Integrity Network (NRIN), this collection contains literature, tools, guidelines, and educational media related to research Integrity. Access the Collection here.

RT2 Los Angeles

BITSS held its Research Transparency and Reproducibility Training (RT2) in Los Angeles, CA, September 5-7, 2018. Find all resources from the training below: Agenda Presentation slides GitHub repository Participant Manual  

Registry of Efficacy and Effectiveness Studies

The Registry of Efficacy and Effectiveness Studies (REES) is a registry for studies designed to establish causal conclusions in Education research. Eligible designs include randomized trials, quasi-experimental designs, regression discontinuity designs, and single-case designs.

RT2 Amsterdam

BITSS held its Research Transparency and Reproducibility Training (RT2) in Amsterdam, the Netherlands, in April 4-6, 2018. Find all resources from the training below: Agenda Participant Manual Presentation slides GitHub repository Final Report  

RT2 London

BITSS held its Research Transparency and Reproducibility Training (RT2) in London, the UK, in September 20-22, 2017. Find all resources from the training below: Agenda Participant Manual Presentation slides Video library GitHub repository Final Report  

RT2 Berkeley

BITSS held its Research Transparency and Reproducibility Training (RT2) in Berkeley, CA, in June 7-9, 2017. Find all resources from the training below: Agenda Participant Manual Presentation slides Video library GitHub repository Final Report  

2016 Summer Institute

BITSS held its third Summer Institute in Berkeley, CA, in June 8-10, 2016. Find all resources from the training below: Agenda Presentation slides GitHub repository Final Report  

2015 Summer Institute

BITSS held its second Summer Institute in Berkeley, CA, in June 10-12, 2015. Find all resources from the training below: Agenda Presentation slides Final Report  

2014 Summer Institute

BITSS held its first Summer Institute in Berkeley, CA, in June 2-6, 2015. Find all resources from the training below: Agenda Presentation slides Final Report

Transparency Training Module for Undergraduate Experimental Economics

These materials were used in the final weeks of an undergraduate course experimental economics at Wesleyan University taught by Professor Jeffrey Naecker. These materials were developed as part of a BITSS Catalyst Training Project “Incorporating Reproducibility and Transparency in an Undergraduate Economics Course” led by Catalyst Jeffrey Naecker.