Welcome to the BITSS Blog

The BITSS blog is an interdisciplinary venue for discussing issues surrounding meta-science including transparency, reproducibility, methodology, ethics, and access. Posts are authored by BITSS staff, as well as affiliated partners and those broadly interested in open science and meta-science.

If you are interested in submitting to the BITSS Blog, please see Guidance for Authors.

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors

  • Star Wars: The Empirics Strike Back

    Guest post by Abel Brodeur, who is joining the economics department at the University of Ottawa   As a visiting Ph.D. student in the economics department at UC Berkeley (2013-2014), I was very fortunate to interact with many professors and Ph.D. students working on research transparency. I realized that several leading researchers in the Bay…

  • News on Transparency in Publication

    Garret Christensen–BITSS Project Scientist   Here are a few quick links related to transparency in academic publication: 1. Finally, it’s not just psych journals that publish registered reports (results-blind review): “Revista de Ciencia Política invites scholars to take part in a special issue on research transparency in the social sciences and specifically in political science.The…

  • Who Inspired the Leamer-Rosenthal Prizes? PART I: Robert Rosenthal

    by Alex Grossman In 1955, Robert “Bob” Rosenthal was a student in the Psychology department at the University of California, Los Angeles (he is now on the faculty at UC Riverside). He spent his days thinking about “psychological projection,” a theory about how we see our own experiences in others. So he designed a study.…

  • The BITSS Take on "wormwars" and Replication Writ Large

    Garret Christensen–BITSS Project Scientist If you’re a development economist, or at all interested in research transparency, I assume you’ve heard about the recent deworming replication controversy. (If you were lucky enough to miss “wormwars,” you can catch up with just about every thing with this one set of links on storify.com). Here at BITSS, we’re…

  • Emerging Researcher Perspectives: Presenting a Pre-Analysis Plan at WGAPE in the Early Stages of Experimental Design

    Guest post by Emily A. West, Ph.D. candidate presenter at the Working Group in African Political Economy (WGAPE) Spring 2015 Annual Meeting. I feel lucky to be a graduate student at a time when many in political science are emphasizing two complementary agendas that are helping the discipline evolve. First, opportunities are being created for graduate…

  • Emerging Researcher Perspectives: Replication as a Credible Pre-Analysis Plan

    One of the most important tools for enhancing the credibility of research is the pre-analysis plan, or the PAP. Simply put, we feel more confident in someone’s inferences if we can verify that they weren’t data mining, engaging in motivated reasoning, or otherwise manipulating their results, knowingly or unknowingly. By publishing a PAP before collecting…

  • New York Times Covers TOP Guidlines

    Yesterday in Science, the Transparency and Openness Promotion (TOP) Committee published the TOP Guidelines, referred to by the New York Times as “the the most comprehensive guidelines for the publication of studies in basic science to date” (see here). The guidelines are the output of a November 2014 meeting at the Center for Open Science (COS), co-hosted…

  • Transparency and Openness Promotion Guidelines

    By Garret Christensen (BITSS) BITSS is proud to announce the publication of the Transparency and Openness Promotion Guidelines in Science. The Guidelines are a set of standards in eight areas of research publication: Citation Standards Data Transparency Analytic Methods (Code) Transparency Research Materials Transparency Design and Analysis Transparency Preregistration of Sudies Preregistration of Analysis Plans Replication

  • Emerging Researcher Perspectives: Get it Right the First Time!

    Guest post by Olivia D’Aoust, Ph.D. in Economics from Université libre de Bruxelles, and former Fulbright Visiting Ph.D. student at the University of California, Berkeley. As a Fulbright PhD student in development economics from Brussels, my experience this past year on the Berkeley campus has been eye opening. In particular, I discovered a new movement…

  • Advisory Board Established for Project TIER

    Guest post by Richard Ball and Norm Medeiros, co-principal investigators of Project TIER at Haverford College. Project TIER (Teaching Integrity in Empirical Economics) is pleased to announce its newly-established Advisory Board. The advisors – George Alter (ICPSR), J. Scott Long (Indiana University), Victoria Stodden (University of Illinois at Urbana-Champaign), and Justin Wolfers (Peterson Institute/University of Michigan) – will help project directors…

  • New Advisory Board Member: Paul Romer

    BITSS is delighted to announce that we’ve added a new member to our advisory board: economist Paul Romer. Paul is a prominent economic theorist who has made major contributions to our understanding of economic growth, technological change, and urbanization. Paul is currently Professor of Economics at NYU, director of the Marron Institute of Urban Management, and director…

  • Influential Paper on Gay Marriage Might Be Marred by Fraudulent Data

    Harsh scrutiny of an influential political science experiment highlights the importance of transparency in research. The paper, from UCLA graduate student Michael LaCour and Columbia University Professor Donald Green, was published in Science in December 2014. It asserted that short conversations with gay canvassers could not only change people’s minds on a divisive social issue like same-sex marriage, but could…

  • P-values are Just the Tip of the Iceberg

    Roger Peng and Jeffrey Leek of John Hopkins University claim that “ridding science of shoddy statistics will require scrutiny of every step, not merely the last one.” This blog post originally appeared in Nature on April 28, 2015 (see here). There is no statistic more maligned than the P value. Hundreds of papers and blogposts have been…

  • Announcing the Leamer-Rosenthal Prizes for Open Social Science

    New prizes will recognize and reward transparency in social science research. BERKELEY, CA (May 13, 2015) – Transparent research is integral to the validity of science. Openness is especially important in such social science disciplines as economics, political science and psychology, because this research shapes policy and influences clinical practices that affect millions of lives.…

  • Recent BITSS Presentations

    Garret Christensen–BITSS Project Scientist BITSS participated in a pair of conferences/workshops recently that we should probably tell you about. First, BITSS was part of a research transparency conference in Washington DC put together by the Laura and John Arnold Foundation. Many of the presentations from the conference can be found here. The idea was to…

  • A Rough Guide to Spotting Bad Science

    Twelve points that will help separate the science from the pseudoscience (see here).    

  • Arnold Foundation Launches New Evidence-Based Policy Division

    The Coalition for Evidence-Based Policy, equivalent to CEGA’s domestic counterpart and a leading force working to institutionalize evidence-based policy making, will merge with one of its funders, the Laura and John Arnold Foundation (LJAF). Also a funder of BITSS, LJAF will integrate the staff of the Coalition into its newly established Evidence-Based Policy and Innovation division. The mission of…

  • Three Transparency Working Papers You Need to Read

    Garret Christensen, BITSS Project Scientist Several great working papers on transparency and replication in economics have been released in the last few months. Two of them are intended for a symposium in The Journal of Economic Perspectives, to which I am very much looking forward, and are about pre-analysis plans. The first of these, by Muriel…

  • Call for Papers: Working Group in African Political Economy (WGAPE)

    BITSS is co-sponsoring the 4th Working Group in African Political Economy (WGAPE) Annual Meeting, taking place May 29-30 at the Watson Institute for International and Public Affairs, Brown University. WGAPE brings together faculty and advanced graduate students in Economics and Political Science who combine field research experience in Africa with training in political economy methods. Paper…

  • BITSS is hiring!

    BITSS is seeking a Senior Program Manager and Coordinator. Applications for both positions will be reviewed on a rolling basis until May 10 for employment beginning in June/July 2015. The Senior Program Manager will be responsible for the overall management, leadership, and oversight of the initiative. The ideal candidate should have: MA/MS in Economics, Political Science,…

  • Registered Reports to the Rescue?

    After writing an article for The Upshot, Brendan Nyhan (Assistant Professor at Dartmouth) was interviewed by The Washington Post. The original Upshot article advocates for a new publishing structure called Registered Reports (RRs): A research publishing format in which protocols and analysis plans are peer reviewed and registered prior to data collection, then published regardless of the outcome. In the following interview with the Washington…

  • TIER Faculty Fellowships 2015-16

    Richard Ball, Associate Professor of Economics, and Norm Medeiros, Associate Librarian, of Haverford College, are co-principal investigators of Project TIER. They are seeking the first class of TIER Fellows to promote and extend teaching of transparent and reproducible empirical research methods. Project TIER (Teaching Integrity in Empirical Research) is an initiative that promotes training in open…

  • The End of p-values?

    Psychology Professors David Trafimow and Michael Marks of New Mexico State University discuss the implications of banning p-values from appearing in published articles. To combat the practice of p-hacking, the editors of Basic and Applied Social Psychology (BASP) will no longer publish p-values included in articles submitted to the journal. The unprecedented move by the journal’s editorial board signals publishing norms may be changing faster…

  • How NOT to Interpret p-values

    Your dose of BITSS humor, via xkcd.

  • Now Accepting Applications for Summer Institute

    BITSS is pleased to announce it is now accepting applications to attend its 2015 Summer Institute. This year’s workshop entitled “Transparency and Reproducibility Methods for Social Science Research” will be held in Berkeley, June 10-12. The intensive course will provide participants with a thorough overview of best practices for open, reproducible research, allowing them to remain…

  • Research Transparency Meeting with CGD

    By Garret Christensen (BITSS) Though BITSS hopes to increase research transparency across the social sciences, several of us, myself included, have a background in development economics. So we were happy to take part in a meeting last week at the Center for Global Development (CGD) in Washington, DC. In addition to BITSS and CGD, representatives…

  • The Disturbing Influence of Flawed Research on Your Living Habits

    Last year, we featured a story on our blog about the so-called cardiovascular benefits of fish oil, largely based on a seminal research study that had more to do with hearsay than with actual science. After your diet, flawed research is now trying to meddle with your sports life. A Danish study published in the Journal of the American College of…

  • Announcing New Grants for Data Publication!

    The Berkeley Initiative for Transparency in the Social Sciences (BITSS), the Center for Effective Global Action (CEGA), and the Alfred P. Sloan Foundation are pleased to announce the Sloan Grants for Data Publication. Regardless of how transparent or rigorous a study design may be, if the study materials (datasets, code, metadata, etc.) are not made…

  • UC Press Launches New Open Access Publications

    The University of California’s publishing house, UC Press, has announced the launch of two new publications, Collabra and Luminos. Collabra will publish academic articles across many academic disciplines including the life, environmental, social and behavioral sciences. Luminos will publish monographs across all fields of study. As the UC Press website indicates, the publications will “not…

  • Science Magazine Releases Special Issue on Digital Privacy

    Yesterday, January 29th, Science Magazine released a new Special Issue entitled The End of Privacy. In line with its theme, the edition will be made available online at no cost for the first week following publication. Take this chance to look through! For scientists, the vast amounts of data that people shed every day offer great…

  • Announcing the New BITSS Advisory Board

    BITSS is pleased to announce its new Advisory Board! Members include leading academics John Ioannidis (Stanford University, School of Medicine), Matthew Rabin (Harvard University, Economics), Bobbie Spellman (University of Virginia, School of Law), and Arthur Lupia (University of Michigan, Political Science). The Board will complement the work of the Executive Committee by providing strategic guidance for BITSS’ continued development, access to a larger network…

  • Win a prize guessing how much trial registration reduces publication bias!

    Does trial registration make an impact on publication bias? Knowing the answer could earn you a cash prize! Macartan Humphreys (Columbia, Political Science) and collaborators Albert Fang and Grant Gordon are doing research on how publication (and publication bias) changed after the introduction of registration in clinical trials. They also want you to guess what the changes were. The bidder…

  • Join an Open Call on Reproducibility Tomorrow at 11 am (ET)

    BITSS Project Scientist Garret Christensen will be participating in a discussion with the Mozilla Science Lab on reproducibility in research tomorrow at 11 am ET. The call is open to the public. For those interested in joining, more information can be found here.

  • Annual Meeting Speakers Now on YouTube!

    In case you missed the 2014 BITSS Research Transparency Forum, you can watch the presentations of all five speakers featured on the BITSS YouTube Channel and embedded in our Annual Meeting page. The YouTube channel also includes videos with interviews from the BITSS Board on the importance of research transparency.

  • Recap of Research Integrity in Economics Session at ASSA 2015

    By Garret Christensen (BITSS) BITSS just got back from the ASSA conference, the major annual gathering of economists. The conference largely serves to help new PhD economists find jobs, but there are of course sessions of research presentations, a media presence and sometimes big names like the Chair-of-the-Federal-Reserve in attendance. BITSS faculty Ted Miguel presided…

  • Come Learn More About Research Transparency at ASSA/AEA

    If you’re at the ASSA meetings in Boston this weekend, and you are interested in learning more about research transparency, then please stop by booth 127 in the exhibition hall to speak with BITSS and Center for Open Science representatives. Or you can attend our session Monday morning at 10:15am: “Promoting New Norms for Transparency…

  • This Monday at AEA2015: Transparency and Integrity in Economic Research Panel

    This January 5th, 10.15am at the American Economic Association Annual Meeting in Boston, MA (Sheraton Hotel, Commonwealth Room). Session: Promoting New Norms for Transparency and Integrity in Economic Research Presiding: Edward Miguel (UC Berkeley) Panelists: Brian Nosek (University of Virginia): “Scientific Utopia: Improving Openness and Reproducibility in Scientific Research” Richard Ball (Haverford College): “Replicability of Empirical Research:…

  • Scientists Have a Sharing Problem

    Dec 15th Maggie Puniewska posted an article in the Atlantic Magazine summarizing the obstacles preventing researchers from sharing their data. The article asks if “science has traditionally been a field that prizes collaboration […] then why [are] so many scientists stingy with their information.” Puniewska outlines the most cited reasons scientists reframe from sharing their…

  • Reflections on Two Years Promoting Transparency in Research

    By Guillaume Kroll (CEGA) Two years ago, in December 2012, a handful of researchers convened in Berkeley to discuss emerging strategies to increase openness and transparency in social science research. The group’s concerns followed a number of high-level cases of scientific misconduct and unethical practices, particularly in psychology (1,2). As researchers started to question the legitimacy of the…

  • Tomorrow! BITSS Research Transparency Forum to Be Livestreamed

    Can’t attend our Annual Meeting? Not to worry, our Public Conference (Thu, 1.30 PM – 5.00 PM Pacific Time) will be livestreamed. For those who will be joining us virtually, questions can be submitted for the Q&A panel session starting at 4:30 PM via Twitter using #bitss2014. All other participants who will be able to attend the Public Conference, the Collaborative…

  • Psychology’s Credibility Crisis

    In a recent interview appearing in Discover Magazine, Brian Nosek, Co-founder of the Center for Open Science and speaker at the upcoming BITSS Annual Meeting, discusses the credibility crisis in psychology.  According to the article, Psychology has lost much of it credibility after a series of published papers were revealed as fraudulent and many other study results were found…

  • Psychology's Credibility Crisis

    In a recent interview appearing in Discover Magazine, Brian Nosek, Co-founder of the Center for Open Science and speaker at the upcoming BITSS Annual Meeting, discusses the credibility crisis in psychology.  According to the article, Psychology has lost much of it credibility after a series of published papers were revealed as fraudulent and many other study results were found…

  • Tools for Research Transparency: a Preview of Upcoming BITSS Training

    By Garret Christensen (BITSS) What are the tools you use to make your research more transparent and reproducible? A lot of my time at BITSS has been spent working on a manual of best practices, and that has required me to familiarize myself with computing tools and resources that make transparent work easier. I’ll be sharing…

  • Scientific Irreproducibility and the Prospects of Meta-Research

    A recent article from The Economist featuring John Ioannidis’ Meta-Research Innovation Center (METRICS), whose work to advance the credibility of research will be presented next week at the BITSS Annual Meeting. “Why most published research findings are false” is not, as the title of an academic paper, likely to win friends in the ivory tower. But it has certainly influenced people (including…

  • Facilitating Radical Change in Publication Standards: Overview of COS Meeting Part II

    Originally posted on the Open Science Collaboration by Denny Borsboom This train won’t stop anytime soon. That’s what I kept thinking during the two-day sessions in Charlottesville, where a diverse array of scientific stakeholders worked hard to reach agreement on new journal standards for open and transparent scientific reporting. The aspired standards are intended to specify practices…

  • Former BITSS Institute Participant Advocates for Replication in Brazil

    Dalson Britto Figueiredo Filho, Adjunct Professor of Political Science at the Federal University of Pernambuco in Recife, Brazil, who attended the BITSS Summer Institute in June 2014, recently published a paper on the importance of replications in Revista Política Hoje. “The BITSS experience really changed my mind on how to do good science”, said Figueiredo Filho. “Now I am…

  • Paper Presentations for Annual Meeting Confirmed!

    With the 2014 Research Transparency Forum around the corner (Dec. 11-12), we are excited to announce the papers to be presented during Friday’s Research Seminar. After carefully reviewing over 30 competitive submissions, BITSS has selected 6 paper presentations: Neil Malhotra (Stanford University): “Publication Bias in the Social Sciences: Unlocking the File Drawer” Uri Simonsohn (University of…

  • Creating Standards for Reproducible Research: Overview of COS Meeting

    By Garret Christensen (BITSS) Representatives from BITSS (CEGA Faculty Director Ted Miguel, CEGA Executive Director Temina Madon, and BITSS Assistant Project Scientist Garret Christensen–that’s me) spent Monday and Tuesday of this week at a very interesting workshop at the Center for Open Science aimed at creating standards for promoting reproducible research in the social-behavioral sciences. Perhaps the…

  • What to Do If You Are Accused of P-Hacking

    In a recent post on Data Colada, University of Pennsylvania Professor Uri Simonsohn discusses what do in the event you (a researcher) are accused of having altered your data to increase statistical significance. Simonsohn states: It has become more common to publicly speculate, upon noticing a paper with unusual analyses, that a reported finding was obtained via p-hacking. For…

  • First Swedish Graduate Student Training in Transparency in the Social Sciences

    Guest Post by Anja Tolonen (University of Gothenburg, Sweden) Seventeen excited graduate students in Economics met at the University of Gothenburg, a Monday in September, to initiate an ongoing discussion about transparency practices in Economics. The students came from all over the world: from Kenya, Romania, Hong Kong, Australia and Sweden of course. The initiative itself also came from…