Resource Library

The BITSS Resource Library contains resources for learning, teaching, and practicing research transparency and reproducibility, including curricula, slide decks, books, guidelines, templates, software, and other tools. All resources are categorized by i) topic, ii) type, and iii) discipline. Filter results by applying criteria along these parameters or use the search bar to find what you’re looking for.

Know of a great resource that we haven’t included or have questions about the existing resources? Email us!

  • Topic

  • Type

  • Discipline

39 Results

ResearchBox Data Management and De-identification+

ResearchBox offers an easy way to share and access scientific content, such as data, code, pre-registrations, and study materials. Uploaded files are organized into “Bingo Tables” that allow readers to easily find & access available files (e.g., researchbox.org/15). Among many features, ResearchBox provides:
  • One-click downloads
  • Instantaneous file-previews
  • Codebooks for every dataset
  • Integration with AsPredicted.org
Read More →

Open Research Calendar Data Management and De-identification+

Open Research Calendar is an open-source community tool that collates information on worldwide events related to open science and research.

CRediT (Contributor Roles Taxonomy) InterdisciplinaryTransparent Reporting

CRediT (Contributor Roles Taxonomy) is high-level taxonomy, including 14 roles, that can be used to represent the roles typically played by contributors to scientific scholarly output. The roles describe each contributor’s specific contribution to the scholarly output.

Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Educational Expansion Epidemiology+

Created by Catalyst Melissa Sharp, this is an open-source repository for epidemiological research methods and reporting skills for observational studies, structured based on the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement. Use it to discover new methods and reporting guidelines and contribute through the GitHub repository (https://github.com/sharpmel/STROBECourse/).

Read More →

Videos: Research Transparency and Reproducibility Training (RT2) – Washington, D.C. Data Management and De-identification+

BITSS hosted a Research Transparency and Reproducibility Training (RT2) in Washington DC, September 11-13, 2019. This was the eighth training event of this kind organized by BITSS since 2014.

RT2 provides participants with an overview of tools and best practices for transparent and reproducible social science research. Click here to videos of presentations given during the training. Find slide decks and other useful materials on this OSF project page (https://osf.io/3mxrw/).

Read More →

ResonsibleData.io Data Management and De-identification+

Using data for social change work offers many opportunities, but it brings challenges, too. The RD community develops practical ways to deal with the unintended consequences of using data in social change work, establishes best practices, and shares approaches between leading thinkers and doers from different sectors. We discuss thorny topics in-person, facilitate online group discussions on the RD mailing list, and share resources on this site.

Read More →

Registry for International Development Impact Evaluations (RIDIE) Economics and Finance+

Administered by the International Initiative for Impact Evaluation (3ie), the Registry for International Development Impact Evaluations (RIDIE) is a registry of impact evaluations related to development in low and middle income countries. RIDIE will register any development impact evaluation that rigorously attempts to estimate the causal impacts of a program, including but not limited to randomized control trials. It is intended to be a prospective registry in which researchers and evaluators can record information about their evaluation designs before conducting the analysis, as well as update information as the study proceeds and post findings upon study completion.

Read More →

Whole Tale Data Management and De-identification+

Whole Tale is an infrastructure that allows users to share data, methods and analysis protocols, and final research outputs in a single, executable object (“living publication” or “tale”) alongside any research publication. Learn more here.

Registered Reports at the Journal of Development Economics Economics and Finance+

As part of a pilot project, the Journal of Development Economics (JDE) now offers authors the opportunity to submit empirical research designs for review and approval before the results of the study are known. The pre-results review track is designed to award well-designed and well-executed studies regardless of whether their empirical results yield clear interpretations.

Learn more about the pilot in this blog post by JDE Editors Andrew Foster and Dean Karlan, and BITSS Faculty Director Edward Miguel.

Read More →

COS Registered Reports information portal Interdisciplinary+

The Center for Open Science (COS) has put together a portal containing information about the registered reports format of peer review and publication. The portal includes general information about registered reports, a list of journals that have implemented the format, an explanation of an appropriate workflow, resources for journal editors, motivation for funders, FAQs, and a list of allied initiatives, inlcuding those that focus on results-blind review and Exploratory Reports.

Read More →

AsPredicted.org Interdisciplinary+

AsPredicted.org is “a standardized pre-registration that requires only what’s necessary to separate exploratory from confirmatory analyses.” You can easily generate a short  and simple pre-registration document that “takes less effort to evaluate than it takes to evaluate the published study itself.” The form, designed by Uri Simonsohn, Joe Simmons, and Leif Nelson, has only nine questions, which are general enough that they are relevant to nearly all disciplines and types of research.

Read More →

TOP Guidelines InterdisciplinaryTransparent Reporting

Transparency and Openness Promotion (TOP) Guidelines are a set eight modular transparency standards for academic journals, each with three levels of increasing stringency. Journals select which of the eight transparency standards they wish to adopt for their journal, and select a level of implementation for the selected standards. These features provide flexibility for adoption depending on disciplinary variation, but simultaneously establish community standards.

Read More →

EQUATOR Network Health Sciences+

The EQUATOR Network is an organization that tracks reporting guidelines across numerous types of studies. They currently have suggestions for over 275 guidelines depending on what type of research you are engaged in.

PRISMA Interdisciplinary+

PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.

 

SPARC (Scholarly Publishing and Academic Resources Coalition) Data Management and De-identification+

This community resource for tracking, comparing, and understanding both current and future U.S. federal funder research data sharing policies is a joint project of SPARC & Johns Hopkins University Libraries.

Accountable Replications Policy “Pottery Barn” Dynamic Documents and Coding Practices+

The Accountable Replication Policy commits the Psychology and Cognitive Neuroscience section of Royal Society Open Science to publishing replications of studies previously published within the journal. Authors can either submit a replication study that is already completed or a proposal to replicate a previous study. To ensure that the review process is unbiased by the results, submissions will be reviewed with existing results initially redacted (where applicable), or in the case of study proposals, before the results exist. Submissions that report close, clear and valid replications of the original methodology will be offered in principle acceptance, which virtually guarantees publication of the replication regardless of the study outcome.

 

Read More →

Go Fishing App Interdisciplinary+

If you get to choose your tests after you see the data, you can get whatever results you like. To see the logic try out this fishy test.

statcheck Wep App Interdisciplinary+

statcheck is a program that checks for errors in statistical reporting in APA-formatted documents. It was originally written in the R programming language. statcheck/web is a web-based implementation of statcheck. Using statcheck/web, you can check any PDF for statistical errors without installing the R programming language on your computer.

Gates Open Research Health Sciences+

Gates Open Research is a scholarly publishing platform that makes research funded by the Bill & Melinda Gates Foundation available quickly and in a format supporting research integrity, reproducibility and transparency. Its open access model enables immediate publication followed by open, invited peer review, combined with an open data policy.

Retraction Watch Interdisciplinary+

Retraction Watch is a blog that reports on retractions of scientific papers, as a window into the scientific process.

Nicebread Data Management and De-identification+

Dr. Felix Schönbrodt’s blog promoting research transparency and open science.

NeuroChambers Issues with transparency and reproducibility+

Chris Chambers is a psychologist and neuroscientist at the School of Psychology, Cardiff University. He created this blog after taking part in a debate about science journalism at the Royal Institution in March 2012. The aim of his blog is give you some insights from the trenches of science. He talks about a range of science-related issues and may even give up a trade secret or two.

 

 

Read More →

Datavyu Data Management and De-identification+

An Open Data Library for Developmental Science that allows users to decide how they want to code video, audio, physiology, motion, or eye tracking data. Power users can write scripts in the Ruby programming language to extend Datavyu’s functionality.

 

Databrary Data Management and De-identification+

Databrary is a video data library for developmental science. Anyone collecting shareable research data will be able to store and organize their data within Databrary after completing the registration process.

rpsychologist Data Management and De-identification+

Kristoffer Magnusson’s blog about R, Statistics, Psychology, Open Science, and Data Visualization.

DMAS Economics and Finance+

The Distributed Meta-Analysis System is an online tool to help scientists analyze, explore, combine, and communicate results from existing empirical studies. It’s primary purpose it to support meta-analyses, by providing a database for empirically estimated models and methods to integrate their results. The current version supports a range of tools that are useful for analyzing empirical climate impact results, but it’s creators intend to expand its applicability to other fields including social sciences, medicine, ecology, and geophysics.

Read More →

Metalab Data Visualization+

MetaLab is a research tool for aggregating across studies in the language acquisition literature. Currently, MetaLab contains 887 effect sizes across meta-analyses in 13 domains of language acquisition, based on data from 252 papers collecting 11363 subjects. These studies can be used to obtain better estimates of effect sizes across different domains, methods, and ages. Using our power calculator, researchers can use these estimates to plan appropriate sample sizes for prospective studies. More generally, MetaLab can be used as a theoretical tool for exploring patterns in development across language acquisition domains.

Read More →

SocArxiv Interdisciplinary+

Open archive of preprints for the social sciences.

Qualitative Data Repository Data Management and De-identification+

QDR selects, ingests, curates, archives, manages, durably preserves, and provides access to digital data used in qualitative and multi-method social inquiry. The repository develops and publicizes common standards and methodologically informed practices for these activities, as well as for the reusing and citing of qualitative data. Four beliefs underpin the repository’s mission: data that can be shared and reused should be; evidence-based claims should be made transparently; teaching is enriched by the use of well-documented data; and rigorous social science requires common understandings of its research methods.

Read More →

AidGrade Economics and Finance+

AidGrade conducts open, real-time meta-analyses of impact evaluations on economic development programs. They collect results from impact evaluations and transparently synthesize them, building a database anyone can use.

Curate Science Issues with transparency and reproducibility+

Curate Science is a crowd-sourced platform to track, organize, and interpret replications of published findings in the social sciences. Curated replication study characteristics include links to PDFs, open/public data, open/public materials, pre-registered protocols, independent variables (IVs), outcome variables (DVs), replication type, replication design differences, and links to associated evidence collections that feature meta-analytic forest plots.

Read More →

Political Science Replication Dynamic Documents and Coding Practices+

Political Science Replication is a blog about reproducibility, replication, pre-registration, research transparency and open peer review.

Replication Network Economics and Finance+

The Replication Network is a group of economists dedicated to promoting the practice of replication in the field of economics.

Replication Wiki Economics and Finance+

ReplicationWiki is a wiki-based service which lists and provides links to replications of empirical studies in economics, studies which have yet to be replicated, and material to assist with replication.

Impact Evaluation Replication Programme Economics and Finance+

International Initiative for Impact Evaluation (3ie) Replication Grant funds replications. Funding requests are reviewed on a rolling basis. High quality applicants are invited to submit full proposals.

Pre-Analysis Plan Checklist Economics and Finance+

Pre-analysis Plan Checklist, by David McKenzie, Lead Economist at the World Bank Development Research Group.