Welcome to the BITSS Blog

BITSS’ Take on Star Wars: The Force Awakens

No, not really, but it does seem that the American Economic Association has fairly good timing with the publication of Brodeur, Lé, Sangnier, and Zylberberg’s “Star Wars: The Empirics Strikes Back” as the lead article in the new issue of American Economic Journal: Applied Economics, which appeared in my inbox this morning. We…

Searching 30,000 Psychology Articles for Statistical Errors

I recently came across this interesting article in psychology, which scanned over 30,000 articles in psychology for statistical reporting errors–checking test statistics against p-values, as some articles claim significance levels that don’t match up with their test statistics. The lead author, Michèle B. Nuijten, blogged about the article for Retraction Watch, where…

BITSS 2015 Annual Meeting NOW!

The BITSS 2015 Annual Meeting is going on now (Thursday and Friday Dec 10-11, at the Magnes in downtown Berkeley). The agenda is here. Tweet about it using #BITSS2015. We’re awarding the Leamer-Rosenthal Prizes for Open Social Science. The prizes have been covered by NYMag and the Atlantic! Materials (slides and such)…

BITSS Sessions Around the World

Garret Christensen–BITSS Project Scientist I’ve recently had the opportunity to represent BITSS at a few interesting meetings and conferences that you might be interested to hear about. A group of political scientists and other social scientists met at Stanford and held a daylong workshop to discuss steps the discipline could take to…

SSMART Grant Recipients

We’re very happy to announce the winners of our SSMART grants! See here for more information, and remember to register for our annual meeting, where several of the winners will be presenting brief summaries of the research they plan to do.

2015 Annual Meeting Agenda

We’ve posted the final agenda for our annual meeting which is coming up Dec 10-11 in Berkeley. Registration is free and open to the public. We’re pretty excited about the lineup, and we hope you’ll join us.

DART Statement Pushback and Response

Garret Christensen–BITSS Project Scientist Hopefully you’re familiar with the DART Statement, a set of recommendations on Data Access and Research Transparency from APSA. (If you’re not, it’s basically political science journals getting together and saying authors will have to share their data in a trusted repository in order to be published.) There’s…

BITSS Guest Post on The Replication Network

Garret Christensen–BITSS Project Scientist I wrote a short guest piece on The Replication Network’s blog to introduce BITSS. It’s a compact description of what we’ve been up to lately if you’re interested.

Open Science Framework Presentation

Courtney Soderberg from the Center for Open Science will be at UC Berkeley for a few days to lead a few workshops that we think you might be interested in. The D-Lab is hosting one of these workshops on November 18: Open Science Framework: Increasing Reproducibility Across the Entire Research Lifecycle. We…

Replication and Transparency Workshop Jan 6-7, 2016

Garret Christensen–BITSS Project Scientist BITSS is happy to announce a workshop on replication and transparency coming up in January, 2016 right after the AEA Annual Meeting in San Francisco, CA. Before I get to that, a reminder about our workshops in November and December. In November, Nicole Janz (of Cambridge University and…

Links: Blind Analysis and Pre-Analysis Plans, Replication Failure

Garret Christensen–BITSS Project Scientist   There’s an interesting new proposal to deal with the problems of bias, p-hacking, and reproducibility failures from Saul Perlmutter, Nobel laureate UC Berkeley physicist and director of the Berkeley Institute for Data Science (where I’m a fellow) called blind analysis. Perlmutter and Robert MacCoun of Stanford have…

Replication in Economics

Garret Christensen–BITSS Project Scientist   CEGA faculty director Ted Miguel was quoted in a Wall Street Journal blog post by Anna Louie Sussman today: “At this point, everybody doing with [sic] work with data and economics has an expectation that their data is very likely to get posted online, that someone is…

SSMART Grants!

Remember how a couple months ago, BITSS announced that we had launched a prize for especially transparent research and teaching? (Read this if you missed it.) Well, today we’re announcing that we’re also looking to fund research on transparency. Of course, you have to do the funded research transparently, meaning that you’ll be…

Workshops in Kenya on Research Transparency

Garret Christensen–BITSS Project Scientist   BITSS has recently launched an initiative to expand research transparency efforts in developing countries. I think this is a very good fit for BITSS–we are housed within the Center for Effective Global Action (CEGA), our faculty lead Ted Miguel is a preeminent development economist, our program director…

Reproducibility Project: Psychology Released in Science

Garret Christensen–BITSS Project Scientist A paper I’ve been excited about for a while now has just been published in Science: “Estimating the Reproducibility of Psychological Science” by the Open Science Collaboration. The paper is the work of the Reproducibility Project: Psychology, coordinated by the Center for Open Science (COS) and funded by…

Recent Articles

Garret Christensen–BITSS Project Scientist   Here are a few recent articles regarding research transparency you might find interesting: Science isn’t broken, it’s just difficult, from 538. Features a great visualization you can play with to demonstrate p-hacking. You can “prove” that Republicans are good for the economy. Or Democrats. YMMV. Spilling the…

Call for Papers, BITSS Annual Meeting Dec 10-11, 2015

We are now accepting submissions of papers to be presented at our 2015 Annual Meeting, which will be held Dec 10-11, 2015 in Berkeley, CA. Submissions are due by Friday, October 9. You can find more information here, or go directly to the submission portal here. Basically, we’re looking for the best…

Who Inspired the Leamer-Rosenthal Prizes? Part II – Ed Leamer

Guest post by Edward Leamer, UCLA Professor of Economics & Statistics I became interested in methodological issues as a University of Michigan graduate student from 1967 to 1970, watching the economics faculty build an econometric macro model in the basement of the building (The Michigan Model), and comparing how these same faculty members described…

Save the Date: BITSS Annual Meeting Dec 10-11, 2015

Mark you calendars now: the fourth BITSS Annual Meeting will be held Thursday and Friday December 10th and 11th in Berkeley, CA. Similar to last year, the event will be open to the public, and will feature an open call for papers (coming soon)! We will also feature presentations of research by…

BITSS on OpenCon Webcast

CEGA Executive Director Temina Madon was the speaker for this month’s OpenCon community webcast and did an excellent job discussing BITSS and the movement for transparency in the social sciences. You can watch her presentation below, or you can download her slides here. We livetweeted during her presentation, so you can find…

Star Wars: The Empirics Strike Back

Guest post by Abel Brodeur, who is joining the economics department at the University of Ottawa   As a visiting Ph.D. student in the economics department at UC Berkeley (2013-2014), I was very fortunate to interact with many professors and Ph.D. students working on research transparency. I realized that several leading researchers…

News on Transparency in Publication

Garret Christensen–BITSS Project Scientist   Here are a few quick links related to transparency in academic publication: 1. Finally, it’s not just psych journals that publish registered reports (results-blind review): “Revista de Ciencia Política invites scholars to take part in a special issue on research transparency in the social sciences and specifically…

Who Inspired the Leamer-Rosenthal Prizes? PART I: Robert Rosenthal

by Alex Grossman In 1955, Robert “Bob” Rosenthal was a student in the Psychology department at the University of California, Los Angeles (he is now on the faculty at UC Riverside). He spent his days thinking about “psychological projection,” a theory about how we see our own experiences in others. So he…

The BITSS Take on "wormwars" and Replication Writ Large

Garret Christensen–BITSS Project Scientist If you’re a development economist, or at all interested in research transparency, I assume you’ve heard about the recent deworming replication controversy. (If you were lucky enough to miss “wormwars,” you can catch up with just about every thing with this one set of links on storify.com). Here…

Emerging Researcher Perspectives: Replication as a Credible Pre-Analysis Plan

One of the most important tools for enhancing the credibility of research is the pre-analysis plan, or the PAP. Simply put, we feel more confident in someone’s inferences if we can verify that they weren’t data mining, engaging in motivated reasoning, or otherwise manipulating their results, knowingly or unknowingly. By publishing a…

New York Times Covers TOP Guidlines

Yesterday in Science, the Transparency and Openness Promotion (TOP) Committee published the TOP Guidelines, referred to by the New York Times as “the the most comprehensive guidelines for the publication of studies in basic science to date” (see here). The guidelines are the output of a November 2014 meeting at the Center for Open…

Transparency and Openness Promotion Guidelines

By Garret Christensen (BITSS) BITSS is proud to announce the publication of the Transparency and Openness Promotion Guidelines in Science. The Guidelines are a set of standards in eight areas of research publication: Citation Standards Data Transparency Analytic Methods (Code) Transparency Research Materials Transparency Design and Analysis Transparency Preregistration of Sudies Preregistration of…

Emerging Researcher Perspectives: Get it Right the First Time!

Guest post by Olivia D’Aoust, Ph.D. in Economics from Université libre de Bruxelles, and former Fulbright Visiting Ph.D. student at the University of California, Berkeley. As a Fulbright PhD student in development economics from Brussels, my experience this past year on the Berkeley campus has been eye opening. In particular, I discovered…

Advisory Board Established for Project TIER

Guest post by Richard Ball and Norm Medeiros, co-principal investigators of Project TIER at Haverford College. Project TIER (Teaching Integrity in Empirical Economics) is pleased to announce its newly-established Advisory Board. The advisors – George Alter (ICPSR), J. Scott Long (Indiana University), Victoria Stodden (University of Illinois at Urbana-Champaign), and Justin Wolfers (Peterson Institute/University of Michigan) – will…

New Advisory Board Member: Paul Romer

BITSS is delighted to announce that we’ve added a new member to our advisory board: economist Paul Romer. Paul is a prominent economic theorist who has made major contributions to our understanding of economic growth, technological change, and urbanization. Paul is currently Professor of Economics at NYU, director of the Marron Institute of Urban…

Influential Paper on Gay Marriage Might Be Marred by Fraudulent Data

Harsh scrutiny of an influential political science experiment highlights the importance of transparency in research. The paper, from UCLA graduate student Michael LaCour and Columbia University Professor Donald Green, was published in Science in December 2014. It asserted that short conversations with gay canvassers could not only change people’s minds on a divisive social issue like same-sex…

P-values are Just the Tip of the Iceberg

Roger Peng and Jeffrey Leek of John Hopkins University claim that “ridding science of shoddy statistics will require scrutiny of every step, not merely the last one.” This blog post originally appeared in Nature on April 28, 2015 (see here). There is no statistic more maligned than the P value. Hundreds of papers and…

Announcing the Leamer-Rosenthal Prizes for Open Social Science

New prizes will recognize and reward transparency in social science research. BERKELEY, CA (May 13, 2015) – Transparent research is integral to the validity of science. Openness is especially important in such social science disciplines as economics, political science and psychology, because this research shapes policy and influences clinical practices that affect…

Recent BITSS Presentations

Garret Christensen–BITSS Project Scientist BITSS participated in a pair of conferences/workshops recently that we should probably tell you about. First, BITSS was part of a research transparency conference in Washington DC put together by the Laura and John Arnold Foundation. Many of the presentations from the conference can be found here. The…

Arnold Foundation Launches New Evidence-Based Policy Division

The Coalition for Evidence-Based Policy, equivalent to CEGA’s domestic counterpart and a leading force working to institutionalize evidence-based policy making, will merge with one of its funders, the Laura and John Arnold Foundation (LJAF). Also a funder of BITSS, LJAF will integrate the staff of the Coalition into its newly established Evidence-Based Policy and Innovation division.…

Three Transparency Working Papers You Need to Read

Garret Christensen, BITSS Project Scientist Several great working papers on transparency and replication in economics have been released in the last few months. Two of them are intended for a symposium in The Journal of Economic Perspectives, to which I am very much looking forward, and are about pre-analysis plans. The first of…

Call for Papers: Working Group in African Political Economy (WGAPE)

BITSS is co-sponsoring the 4th Working Group in African Political Economy (WGAPE) Annual Meeting, taking place May 29-30 at the Watson Institute for International and Public Affairs, Brown University. WGAPE brings together faculty and advanced graduate students in Economics and Political Science who combine field research experience in Africa with training in…

BITSS is hiring!

BITSS is seeking a Senior Program Manager and Coordinator. Applications for both positions will be reviewed on a rolling basis until May 10 for employment beginning in June/July 2015. The Senior Program Manager will be responsible for the overall management, leadership, and oversight of the initiative. The ideal candidate should have: MA/MS in…

Registered Reports to the Rescue?

After writing an article for The Upshot, Brendan Nyhan (Assistant Professor at Dartmouth) was interviewed by The Washington Post. The original Upshot article advocates for a new publishing structure called Registered Reports (RRs): A research publishing format in which protocols and analysis plans are peer reviewed and registered prior to data collection, then published regardless of the outcome. In the following interview…

TIER Faculty Fellowships 2015-16

Richard Ball, Associate Professor of Economics, and Norm Medeiros, Associate Librarian, of Haverford College, are co-principal investigators of Project TIER. They are seeking the first class of TIER Fellows to promote and extend teaching of transparent and reproducible empirical research methods. Project TIER (Teaching Integrity in Empirical Research) is an initiative that promotes…

The End of p-values?

Psychology Professors David Trafimow and Michael Marks of New Mexico State University discuss the implications of banning p-values from appearing in published articles. To combat the practice of p-hacking, the editors of Basic and Applied Social Psychology (BASP) will no longer publish p-values included in articles submitted to the journal. The unprecedented move by the journal’s editorial board signals publishing norms may…

Now Accepting Applications for Summer Institute

BITSS is pleased to announce it is now accepting applications to attend its 2015 Summer Institute. This year’s workshop entitled “Transparency and Reproducibility Methods for Social Science Research” will be held in Berkeley, June 10-12. The intensive course will provide participants with a thorough overview of best practices for open, reproducible research, allowing…

Research Transparency Meeting with CGD

By Garret Christensen (BITSS) Though BITSS hopes to increase research transparency across the social sciences, several of us, myself included, have a background in development economics. So we were happy to take part in a meeting last week at the Center for Global Development (CGD) in Washington, DC. In addition to BITSS…

The Disturbing Influence of Flawed Research on Your Living Habits

Last year, we featured a story on our blog about the so-called cardiovascular benefits of fish oil, largely based on a seminal research study that had more to do with hearsay than with actual science. After your diet, flawed research is now trying to meddle with your sports life. A Danish study published in the Journal of the…

Announcing New Grants for Data Publication!

The Berkeley Initiative for Transparency in the Social Sciences (BITSS), the Center for Effective Global Action (CEGA), and the Alfred P. Sloan Foundation are pleased to announce the Sloan Grants for Data Publication. Regardless of how transparent or rigorous a study design may be, if the study materials (datasets, code, metadata, etc.)…

UC Press Launches New Open Access Publications

The University of California’s publishing house, UC Press, has announced the launch of two new publications, Collabra and Luminos. Collabra will publish academic articles across many academic disciplines including the life, environmental, social and behavioral sciences. Luminos will publish monographs across all fields of study. As the UC Press website indicates, the…