Welcome to the BITSS Blog

Call for Cases of Data Reuse: Still Seeking Answers

Guest post by Stephanie Wykstra, Innovations for Poverty Action         As advocates for open data, my colleagues and I often point to re-use of data for further research as a major benefit of data-sharing. In fact there are many cases in which shared data was clearly very useful for…

BITSS Bits: Repro at UC Davis and the World Bank, Job Openings

–Garret Christensen, BITSS Project Scientist BITSS Bits. Too cheesy for our newsletter name? My guess is yes. Regardless, here’s an update on recent BITSS activities. I presented on p-hacking, publication bias, and pre-registration at the Making Social Science Transparent conference at the UC Davis Institute for Social Sciences April 22. My slides…

BITSS and BIDS Collaboration: Call for Reproducible Workflows

BITSS and the Reproducibility Working Group at the Berkeley Institute for Data Science are collaborating on an edited volume of reproducible workflows in the social sciences, and we are looking for submissions. BIDS Fellow Cyrus Dioun wrote about it on the Bad Hessian computational social science blog: “[M]aking work reproducible can feel…

Upcoming Events and Links

Fitsum Mulugeta, a former EASST fellow who attended the IPA/BITSS transparency workshop in Kenya last month, wrote a blog post about his experience. David Broockman and Joshua Kalla (Leamer-Rosenthal prize winners) got more press coverage about their paper in Science on voter persuasion, this time from This American Life. Ted Miguel will…

Research Transparency Job Openings

Both Innovations for Poverty Action (IPA) and BITSS are hiring in the area of research transparency. IPA is hiring a coordinator, and BITSS is looking for a new program manager. Please apply and/or share widely.

Leamer-Rosenthal Prize Winners in the News Again

Last year BITSS gave some of our first Leamer-Rosenthal Prizes for Open Social Science to David Broockman and Joshua Kalla. They just had a new paper published in Science, and are keeping up the transparency–the supplementary materials file points clearly to a data, code, and interview scripts archive on Harvard’s Dataverse. Their paper…

“Making Social Science Transparent” Conference at UC Davis April 22

BITSS is taking part in a conference that we wanted to let you know about– The Institute for Social Sciences at UC Davis cordially invites you to attend: “Making Social Science Transparent: A Conference on the Issues of Transparency, Data and Code Accessibility, Replication, and Reproducibility” Friday, April 22, 2016 8:30 a.m.–6:00…

Summer Institute Application Deadline Extended until April 8

We’ve extended the deadline to apply for the BITSS Summer Institute until April 8. The summer institute will be June 8-10 in Berkeley, CA, and will bring together some of the leading researchers in social science and public health working on reproducibility. Read the full description of the event here. We hope…

BITSS/IPA Kenya Workshop on data-sharing and reproducibility

BITSS and Innovations for Poverty Action (IPA) recently co-hosted a workshop on reproducibility outside Nairobi, Kenya. IPA’s Stephanie Wykstra wrote a post for the IPA blog about the event: During the workshop (agenda here) we covered topics on how to make research reproducible. The sessions included an overview of why research is unreliable and…

Summer Reproducibility Workshop Announcements!

BITSS and related organizations have several events coming up. We strongly encourage you to apply, and to please share these announcements with graduate students or researchers in your networks who may be interested in attending. June 8-10 in Berkeley, California,  BITSS is holding its third annual Summer Institute, an intense three-day introduction…

Replication Project: Economics–sort of

Garret Christensen–BITSS Project Scientist I say sort of because it wasn’t run by the Center for Open Science, but in a similar spirit to the Replication Project: Psychology, Colin Camerer led a big reproducibility project related to economics experiments that came out in Science today.  Here’s the related news article, and here’s…

New BITSS Paper on Publication Bias

Garret Christensen–BITSS Project Scientist I’m happy to report that I just published a paper with Justin McCrary and Daniele Fanelli in PLOS ONE on publication bias, “Conservative Tests under Satisficing Models of Publication Bias.” In English, that’s “if I assume there’s a lot of publication bias, what t-statistics should I really interpret…

BITSS Is Hiring Another Scientist

BITSS is hiring another project scientist! Help us develop and launch a big study of researcher practices, and help us promote and develop reproducible research.  The official posting is here.

Project TIER Faculty Fellowship Opportunities

Our friends at Project TIER have a new funding opportunity for faculty interested in teaching reproducible empirical methods: Project TIER is pleased to announce new rounds of two programs for social science faculty interested in training undergraduate and graduate students in the social sciences in transparent and reproducible methods of research. TIER…

More SSMART Funding!

BITSS is delighted to announce the second round of our Social Science Meta-Analysis and Research Transparency (SSMART) grants. The RFP can be found here. The deadline to apply is 11:59 pm US Pacific Time, Monday, March 28, 2016.

Upcoming Events & Recent Papers

Garret Christensen–BITSS Project Scientist   How has registration affected publication bias in medical research? This comes as a surprise to me, but maybe not so much, says Grant M. Gordon. He’s presenting on this research Monday 1/25 in the BIDS space (190 Doe Library) at UC Berkeley. Find the paper and more…

Introducing The Replication Network (TRN)

Guest post by W. Robert Reed, Professor in Economics in the Department of Economics and Finance, University of Canterbury, New Zealand and co-organizer of The Replication Network           Replication is hot. At least I think it must be, because typing “replication AND science” in Google produces hundreds of…

Open Source Software for Reproducible Social Science

Garret Christensen –BITSS Project Scientist   BITSS offers grad student workshops in reproducible research, where we give a hands-on introduction to software that can help make your work more reproducible. A lot of the software is listed on the Software section of our Resources page, but I wanted to create a quick…

Science is “show me,” not “trust me”

Guest post by Philip B. Stark, Associate Dean of the Division of Mathematical and Physical Sciences, UC Berkeley Professor of Statistics, and winner of one of BITSS’ Leamer-Rosenthal Prizes for Open Social Science.         Reproducibility and open science are about providing evidence that you are right, not just claiming…

BITSS’ Take on Star Wars: The Force Awakens

No, not really, but it does seem that the American Economic Association has fairly good timing with the publication of Brodeur, Lé, Sangnier, and Zylberberg’s “Star Wars: The Empirics Strikes Back” as the lead article in the new issue of American Economic Journal: Applied Economics, which appeared in my inbox this morning. We…

Searching 30,000 Psychology Articles for Statistical Errors

I recently came across this interesting article in psychology, which scanned over 30,000 articles in psychology for statistical reporting errors–checking test statistics against p-values, as some articles claim significance levels that don’t match up with their test statistics. The lead author, Michèle B. Nuijten, blogged about the article for Retraction Watch, where…

BITSS 2015 Annual Meeting NOW!

The BITSS 2015 Annual Meeting is going on now (Thursday and Friday Dec 10-11, at the Magnes in downtown Berkeley). The agenda is here. Tweet about it using #BITSS2015. We’re awarding the Leamer-Rosenthal Prizes for Open Social Science. The prizes have been covered by NYMag and the Atlantic! Materials (slides and such)…

BITSS Sessions Around the World

Garret Christensen–BITSS Project Scientist I’ve recently had the opportunity to represent BITSS at a few interesting meetings and conferences that you might be interested to hear about. A group of political scientists and other social scientists met at Stanford and held a daylong workshop to discuss steps the discipline could take to…

SSMART Grant Recipients

We’re very happy to announce the winners of our SSMART grants! See here for more information, and remember to register for our annual meeting, where several of the winners will be presenting brief summaries of the research they plan to do.

2015 Annual Meeting Agenda

We’ve posted the final agenda for our annual meeting which is coming up Dec 10-11 in Berkeley. Registration is free and open to the public. We’re pretty excited about the lineup, and we hope you’ll join us.

DART Statement Pushback and Response

Garret Christensen–BITSS Project Scientist Hopefully you’re familiar with the DART Statement, a set of recommendations on Data Access and Research Transparency from APSA. (If you’re not, it’s basically political science journals getting together and saying authors will have to share their data in a trusted repository in order to be published.) There’s…

BITSS Guest Post on The Replication Network

Garret Christensen–BITSS Project Scientist I wrote a short guest piece on The Replication Network’s blog to introduce BITSS. It’s a compact description of what we’ve been up to lately if you’re interested.

Open Science Framework Presentation

Courtney Soderberg from the Center for Open Science will be at UC Berkeley for a few days to lead a few workshops that we think you might be interested in. The D-Lab is hosting one of these workshops on November 18: Open Science Framework: Increasing Reproducibility Across the Entire Research Lifecycle. We…

Replication and Transparency Workshop Jan 6-7, 2016

Garret Christensen–BITSS Project Scientist BITSS is happy to announce a workshop on replication and transparency coming up in January, 2016 right after the AEA Annual Meeting in San Francisco, CA. Before I get to that, a reminder about our workshops in November and December. In November, Nicole Janz (of Cambridge University and…

Links: Blind Analysis and Pre-Analysis Plans, Replication Failure

Garret Christensen–BITSS Project Scientist   There’s an interesting new proposal to deal with the problems of bias, p-hacking, and reproducibility failures from Saul Perlmutter, Nobel laureate UC Berkeley physicist and director of the Berkeley Institute for Data Science (where I’m a fellow) called blind analysis. Perlmutter and Robert MacCoun of Stanford have…

Replication in Economics

Garret Christensen–BITSS Project Scientist   CEGA faculty director Ted Miguel was quoted in a Wall Street Journal blog post by Anna Louie Sussman today: “At this point, everybody doing with [sic] work with data and economics has an expectation that their data is very likely to get posted online, that someone is…

SSMART Grants!

Remember how a couple months ago, BITSS announced that we had launched a prize for especially transparent research and teaching? (Read this if you missed it.) Well, today we’re announcing that we’re also looking to fund research on transparency. Of course, you have to do the funded research transparently, meaning that you’ll be…

Workshops in Kenya on Research Transparency

Garret Christensen–BITSS Project Scientist   BITSS has recently launched an initiative to expand research transparency efforts in developing countries. I think this is a very good fit for BITSS–we are housed within the Center for Effective Global Action (CEGA), our faculty lead Ted Miguel is a preeminent development economist, our program director…

Reproducibility Project: Psychology Released in Science

Garret Christensen–BITSS Project Scientist A paper I’ve been excited about for a while now has just been published in Science: “Estimating the Reproducibility of Psychological Science” by the Open Science Collaboration. The paper is the work of the Reproducibility Project: Psychology, coordinated by the Center for Open Science (COS) and funded by…

Recent Articles

Garret Christensen–BITSS Project Scientist   Here are a few recent articles regarding research transparency you might find interesting: Science isn’t broken, it’s just difficult, from 538. Features a great visualization you can play with to demonstrate p-hacking. You can “prove” that Republicans are good for the economy. Or Democrats. YMMV. Spilling the…

Call for Papers, BITSS Annual Meeting Dec 10-11, 2015

We are now accepting submissions of papers to be presented at our 2015 Annual Meeting, which will be held Dec 10-11, 2015 in Berkeley, CA. Submissions are due by Friday, October 9. You can find more information here, or go directly to the submission portal here. Basically, we’re looking for the best…

Who Inspired the Leamer-Rosenthal Prizes? Part II – Ed Leamer

Guest post by Edward Leamer, UCLA Professor of Economics & Statistics I became interested in methodological issues as a University of Michigan graduate student from 1967 to 1970, watching the economics faculty build an econometric macro model in the basement of the building (The Michigan Model), and comparing how these same faculty members described…

Save the Date: BITSS Annual Meeting Dec 10-11, 2015

Mark you calendars now: the fourth BITSS Annual Meeting will be held Thursday and Friday December 10th and 11th in Berkeley, CA. Similar to last year, the event will be open to the public, and will feature an open call for papers (coming soon)! We will also feature presentations of research by…

BITSS on OpenCon Webcast

CEGA Executive Director Temina Madon was the speaker for this month’s OpenCon community webcast and did an excellent job discussing BITSS and the movement for transparency in the social sciences. You can watch her presentation below, or you can download her slides here. We livetweeted during her presentation, so you can find…

Star Wars: The Empirics Strike Back

Guest post by Abel Brodeur, who is joining the economics department at the University of Ottawa   As a visiting Ph.D. student in the economics department at UC Berkeley (2013-2014), I was very fortunate to interact with many professors and Ph.D. students working on research transparency. I realized that several leading researchers…

News on Transparency in Publication

Garret Christensen–BITSS Project Scientist   Here are a few quick links related to transparency in academic publication: 1. Finally, it’s not just psych journals that publish registered reports (results-blind review): “Revista de Ciencia Política invites scholars to take part in a special issue on research transparency in the social sciences and specifically…

Who Inspired the Leamer-Rosenthal Prizes? PART I: Robert Rosenthal

by Alex Grossman In 1955, Robert “Bob” Rosenthal was a student in the Psychology department at the University of California, Los Angeles (he is now on the faculty at UC Riverside). He spent his days thinking about “psychological projection,” a theory about how we see our own experiences in others. So he…

The BITSS Take on "wormwars" and Replication Writ Large

Garret Christensen–BITSS Project Scientist If you’re a development economist, or at all interested in research transparency, I assume you’ve heard about the recent deworming replication controversy. (If you were lucky enough to miss “wormwars,” you can catch up with just about every thing with this one set of links on storify.com). Here…

Emerging Researcher Perspectives: Replication as a Credible Pre-Analysis Plan

One of the most important tools for enhancing the credibility of research is the pre-analysis plan, or the PAP. Simply put, we feel more confident in someone’s inferences if we can verify that they weren’t data mining, engaging in motivated reasoning, or otherwise manipulating their results, knowingly or unknowingly. By publishing a…

New York Times Covers TOP Guidlines

Yesterday in Science, the Transparency and Openness Promotion (TOP) Committee published the TOP Guidelines, referred to by the New York Times as “the the most comprehensive guidelines for the publication of studies in basic science to date” (see here). The guidelines are the output of a November 2014 meeting at the Center for Open…

Transparency and Openness Promotion Guidelines

By Garret Christensen (BITSS) BITSS is proud to announce the publication of the Transparency and Openness Promotion Guidelines in Science. The Guidelines are a set of standards in eight areas of research publication: Citation Standards Data Transparency Analytic Methods (Code) Transparency Research Materials Transparency Design and Analysis Transparency Preregistration of Sudies Preregistration of…

Emerging Researcher Perspectives: Get it Right the First Time!

Guest post by Olivia D’Aoust, Ph.D. in Economics from Université libre de Bruxelles, and former Fulbright Visiting Ph.D. student at the University of California, Berkeley. As a Fulbright PhD student in development economics from Brussels, my experience this past year on the Berkeley campus has been eye opening. In particular, I discovered…

Advisory Board Established for Project TIER

Guest post by Richard Ball and Norm Medeiros, co-principal investigators of Project TIER at Haverford College. Project TIER (Teaching Integrity in Empirical Economics) is pleased to announce its newly-established Advisory Board. The advisors – George Alter (ICPSR), J. Scott Long (Indiana University), Victoria Stodden (University of Illinois at Urbana-Champaign), and Justin Wolfers (Peterson Institute/University of Michigan) – will…