–Garret Christensen, BITSS Project Scientist BITSS Bits. Too cheesy for our newsletter name? My guess is yes. Regardless, here’s an update on recent BITSS activities. I presented on p-hacking, publication bias, and pre-registration at the Making Social Science Transparent conference at the UC Davis Institute for Social Sciences April 22. My slides…
BITSS and BIDS Collaboration: Call for Reproducible Workflows
BITSS and the Reproducibility Working Group at the Berkeley Institute for Data Science are collaborating on an edited volume of reproducible workflows in the social sciences, and we are looking for submissions. BIDS Fellow Cyrus Dioun wrote about it on the Bad Hessian computational social science blog: “[M]aking work reproducible can feel…
Upcoming Events and Links
Fitsum Mulugeta, a former EASST fellow who attended the IPA/BITSS transparency workshop in Kenya last month, wrote a blog post about his experience. David Broockman and Joshua Kalla (Leamer-Rosenthal prize winners) got more press coverage about their paper in Science on voter persuasion, this time from This American Life. Ted Miguel will…
Research Transparency Job Openings
Both Innovations for Poverty Action (IPA) and BITSS are hiring in the area of research transparency. IPA is hiring a coordinator, and BITSS is looking for a new program manager. Please apply and/or share widely.
Leamer-Rosenthal Prize Winners in the News Again
Last year BITSS gave some of our first Leamer-Rosenthal Prizes for Open Social Science to David Broockman and Joshua Kalla. They just had a new paper published in Science, and are keeping up the transparency–the supplementary materials file points clearly to a data, code, and interview scripts archive on Harvard’s Dataverse. Their paper…
“Making Social Science Transparent” Conference at UC Davis April 22
BITSS is taking part in a conference that we wanted to let you know about– The Institute for Social Sciences at UC Davis cordially invites you to attend: “Making Social Science Transparent: A Conference on the Issues of Transparency, Data and Code Accessibility, Replication, and Reproducibility” Friday, April 22, 2016 8:30 a.m.–6:00…
Summer Institute Application Deadline Extended until April 8
We’ve extended the deadline to apply for the BITSS Summer Institute until April 8. The summer institute will be June 8-10 in Berkeley, CA, and will bring together some of the leading researchers in social science and public health working on reproducibility. Read the full description of the event here. We hope…
BITSS/IPA Kenya Workshop on data-sharing and reproducibility
BITSS and Innovations for Poverty Action (IPA) recently co-hosted a workshop on reproducibility outside Nairobi, Kenya. IPA’s Stephanie Wykstra wrote a post for the IPA blog about the event: During the workshop (agenda here) we covered topics on how to make research reproducible. The sessions included an overview of why research is unreliable and…
Summer Reproducibility Workshop Announcements!
BITSS and related organizations have several events coming up. We strongly encourage you to apply, and to please share these announcements with graduate students or researchers in your networks who may be interested in attending. June 8-10 in Berkeley, California, BITSS is holding its third annual Summer Institute, an intense three-day introduction…
Replication Project: Economics–sort of
Garret Christensen–BITSS Project Scientist I say sort of because it wasn’t run by the Center for Open Science, but in a similar spirit to the Replication Project: Psychology, Colin Camerer led a big reproducibility project related to economics experiments that came out in Science today. Here’s the related news article, and here’s…
New BITSS Paper on Publication Bias
Garret Christensen–BITSS Project Scientist I’m happy to report that I just published a paper with Justin McCrary and Daniele Fanelli in PLOS ONE on publication bias, “Conservative Tests under Satisficing Models of Publication Bias.” In English, that’s “if I assume there’s a lot of publication bias, what t-statistics should I really interpret…
BITSS Is Hiring Another Scientist
BITSS is hiring another project scientist! Help us develop and launch a big study of researcher practices, and help us promote and develop reproducible research. The official posting is here.
Project TIER Faculty Fellowship Opportunities
Our friends at Project TIER have a new funding opportunity for faculty interested in teaching reproducible empirical methods: Project TIER is pleased to announce new rounds of two programs for social science faculty interested in training undergraduate and graduate students in the social sciences in transparent and reproducible methods of research. TIER…
More SSMART Funding!
BITSS is delighted to announce the second round of our Social Science Meta-Analysis and Research Transparency (SSMART) grants. The RFP can be found here. The deadline to apply is 11:59 pm US Pacific Time, Monday, March 28, 2016.
Upcoming Events & Recent Papers
Garret Christensen–BITSS Project Scientist How has registration affected publication bias in medical research? This comes as a surprise to me, but maybe not so much, says Grant M. Gordon. He’s presenting on this research Monday 1/25 in the BIDS space (190 Doe Library) at UC Berkeley. Find the paper and more…
Introducing The Replication Network (TRN)
Guest post by W. Robert Reed, Professor in Economics in the Department of Economics and Finance, University of Canterbury, New Zealand and co-organizer of The Replication Network Replication is hot. At least I think it must be, because typing “replication AND science” in Google produces hundreds of…
Open Source Software for Reproducible Social Science
Garret Christensen –BITSS Project Scientist BITSS offers grad student workshops in reproducible research, where we give a hands-on introduction to software that can help make your work more reproducible. A lot of the software is listed on the Software section of our Resources page, but I wanted to create a quick…
Science is “show me,” not “trust me”
Guest post by Philip B. Stark, Associate Dean of the Division of Mathematical and Physical Sciences, UC Berkeley Professor of Statistics, and winner of one of BITSS’ Leamer-Rosenthal Prizes for Open Social Science. Reproducibility and open science are about providing evidence that you are right, not just claiming…
BITSS’ Take on Star Wars: The Force Awakens
No, not really, but it does seem that the American Economic Association has fairly good timing with the publication of Brodeur, Lé, Sangnier, and Zylberberg’s “Star Wars: The Empirics Strikes Back” as the lead article in the new issue of American Economic Journal: Applied Economics, which appeared in my inbox this morning. We…
Searching 30,000 Psychology Articles for Statistical Errors
I recently came across this interesting article in psychology, which scanned over 30,000 articles in psychology for statistical reporting errors–checking test statistics against p-values, as some articles claim significance levels that don’t match up with their test statistics. The lead author, Michèle B. Nuijten, blogged about the article for Retraction Watch, where…
BITSS 2015 Annual Meeting NOW!
The BITSS 2015 Annual Meeting is going on now (Thursday and Friday Dec 10-11, at the Magnes in downtown Berkeley). The agenda is here. Tweet about it using #BITSS2015. We’re awarding the Leamer-Rosenthal Prizes for Open Social Science. The prizes have been covered by NYMag and the Atlantic! Materials (slides and such)…
Who Inspired the Leamer-Rosenthal Prizes? Part II – Ed Leamer
Guest post by Edward Leamer, UCLA Professor of Economics & Statistics I became interested in methodological issues as a University of Michigan graduate student from 1967 to 1970, watching the economics faculty build an econometric macro model in the basement of the building (The Michigan Model), and comparing how these same faculty members described…
New York Times Covers TOP Guidlines
Yesterday in Science, the Transparency and Openness Promotion (TOP) Committee published the TOP Guidelines, referred to by the New York Times as “the the most comprehensive guidelines for the publication of studies in basic science to date” (see here). The guidelines are the output of a November 2014 meeting at the Center for Open…
Transparency and Openness Promotion Guidelines
By Garret Christensen (BITSS) BITSS is proud to announce the publication of the Transparency and Openness Promotion Guidelines in Science. The Guidelines are a set of standards in eight areas of research publication: Citation Standards Data Transparency Analytic Methods (Code) Transparency Research Materials Transparency Design and Analysis Transparency Preregistration of Sudies Preregistration of…
Emerging Researcher Perspectives: Get it Right the First Time!
Guest post by Olivia D’Aoust, Ph.D. in Economics from Université libre de Bruxelles, and former Fulbright Visiting Ph.D. student at the University of California, Berkeley. As a Fulbright PhD student in development economics from Brussels, my experience this past year on the Berkeley campus has been eye opening. In particular, I discovered…
Advisory Board Established for Project TIER
Guest post by Richard Ball and Norm Medeiros, co-principal investigators of Project TIER at Haverford College. Project TIER (Teaching Integrity in Empirical Economics) is pleased to announce its newly-established Advisory Board. The advisors – George Alter (ICPSR), J. Scott Long (Indiana University), Victoria Stodden (University of Illinois at Urbana-Champaign), and Justin Wolfers (Peterson Institute/University of Michigan) – will…
New Advisory Board Member: Paul Romer
BITSS is delighted to announce that we’ve added a new member to our advisory board: economist Paul Romer. Paul is a prominent economic theorist who has made major contributions to our understanding of economic growth, technological change, and urbanization. Paul is currently Professor of Economics at NYU, director of the Marron Institute of Urban…
Influential Paper on Gay Marriage Might Be Marred by Fraudulent Data
Harsh scrutiny of an influential political science experiment highlights the importance of transparency in research. The paper, from UCLA graduate student Michael LaCour and Columbia University Professor Donald Green, was published in Science in December 2014. It asserted that short conversations with gay canvassers could not only change people’s minds on a divisive social issue like same-sex…
P-values are Just the Tip of the Iceberg
Roger Peng and Jeffrey Leek of John Hopkins University claim that “ridding science of shoddy statistics will require scrutiny of every step, not merely the last one.” This blog post originally appeared in Nature on April 28, 2015 (see here). There is no statistic more maligned than the P value. Hundreds of papers and…
Announcing the Leamer-Rosenthal Prizes for Open Social Science
New prizes will recognize and reward transparency in social science research. BERKELEY, CA (May 13, 2015) – Transparent research is integral to the validity of science. Openness is especially important in such social science disciplines as economics, political science and psychology, because this research shapes policy and influences clinical practices that affect…
Recent BITSS Presentations
Garret Christensen–BITSS Project Scientist BITSS participated in a pair of conferences/workshops recently that we should probably tell you about. First, BITSS was part of a research transparency conference in Washington DC put together by the Laura and John Arnold Foundation. Many of the presentations from the conference can be found here. The…
A Rough Guide to Spotting Bad Science
Twelve points that will help separate the science from the pseudoscience (see here).
Arnold Foundation Launches New Evidence-Based Policy Division
The Coalition for Evidence-Based Policy, equivalent to CEGA’s domestic counterpart and a leading force working to institutionalize evidence-based policy making, will merge with one of its funders, the Laura and John Arnold Foundation (LJAF). Also a funder of BITSS, LJAF will integrate the staff of the Coalition into its newly established Evidence-Based Policy and Innovation division.…
Three Transparency Working Papers You Need to Read
Garret Christensen, BITSS Project Scientist Several great working papers on transparency and replication in economics have been released in the last few months. Two of them are intended for a symposium in The Journal of Economic Perspectives, to which I am very much looking forward, and are about pre-analysis plans. The first of…
Call for Papers: Working Group in African Political Economy (WGAPE)
BITSS is co-sponsoring the 4th Working Group in African Political Economy (WGAPE) Annual Meeting, taking place May 29-30 at the Watson Institute for International and Public Affairs, Brown University. WGAPE brings together faculty and advanced graduate students in Economics and Political Science who combine field research experience in Africa with training in…
BITSS is hiring!
BITSS is seeking a Senior Program Manager and Coordinator. Applications for both positions will be reviewed on a rolling basis until May 10 for employment beginning in June/July 2015. The Senior Program Manager will be responsible for the overall management, leadership, and oversight of the initiative. The ideal candidate should have: MA/MS in…
Registered Reports to the Rescue?
After writing an article for The Upshot, Brendan Nyhan (Assistant Professor at Dartmouth) was interviewed by The Washington Post. The original Upshot article advocates for a new publishing structure called Registered Reports (RRs): A research publishing format in which protocols and analysis plans are peer reviewed and registered prior to data collection, then published regardless of the outcome. In the following interview…
TIER Faculty Fellowships 2015-16
Richard Ball, Associate Professor of Economics, and Norm Medeiros, Associate Librarian, of Haverford College, are co-principal investigators of Project TIER. They are seeking the first class of TIER Fellows to promote and extend teaching of transparent and reproducible empirical research methods. Project TIER (Teaching Integrity in Empirical Research) is an initiative that promotes…
The End of p-values?
Psychology Professors David Trafimow and Michael Marks of New Mexico State University discuss the implications of banning p-values from appearing in published articles. To combat the practice of p-hacking, the editors of Basic and Applied Social Psychology (BASP) will no longer publish p-values included in articles submitted to the journal. The unprecedented move by the journal’s editorial board signals publishing norms may…
How NOT to Interpret p-values
Your dose of BITSS humor, via xkcd.
Now Accepting Applications for Summer Institute
BITSS is pleased to announce it is now accepting applications to attend its 2015 Summer Institute. This year’s workshop entitled “Transparency and Reproducibility Methods for Social Science Research” will be held in Berkeley, June 10-12. The intensive course will provide participants with a thorough overview of best practices for open, reproducible research, allowing…
Research Transparency Meeting with CGD
By Garret Christensen (BITSS) Though BITSS hopes to increase research transparency across the social sciences, several of us, myself included, have a background in development economics. So we were happy to take part in a meeting last week at the Center for Global Development (CGD) in Washington, DC. In addition to BITSS…
The Disturbing Influence of Flawed Research on Your Living Habits
Last year, we featured a story on our blog about the so-called cardiovascular benefits of fish oil, largely based on a seminal research study that had more to do with hearsay than with actual science. After your diet, flawed research is now trying to meddle with your sports life. A Danish study published in the Journal of the…
Announcing New Grants for Data Publication!
The Berkeley Initiative for Transparency in the Social Sciences (BITSS), the Center for Effective Global Action (CEGA), and the Alfred P. Sloan Foundation are pleased to announce the Sloan Grants for Data Publication. Regardless of how transparent or rigorous a study design may be, if the study materials (datasets, code, metadata, etc.)…
UC Press Launches New Open Access Publications
The University of California’s publishing house, UC Press, has announced the launch of two new publications, Collabra and Luminos. Collabra will publish academic articles across many academic disciplines including the life, environmental, social and behavioral sciences. Luminos will publish monographs across all fields of study. As the UC Press website indicates, the…
Science Magazine Releases Special Issue on Digital Privacy
Yesterday, January 29th, Science Magazine released a new Special Issue entitled The End of Privacy. In line with its theme, the edition will be made available online at no cost for the first week following publication. Take this chance to look through! For scientists, the vast amounts of data that people shed every…
Announcing the New BITSS Advisory Board
BITSS is pleased to announce its new Advisory Board! Members include leading academics John Ioannidis (Stanford University, School of Medicine), Matthew Rabin (Harvard University, Economics), Bobbie Spellman (University of Virginia, School of Law), and Arthur Lupia (University of Michigan, Political Science). The Board will complement the work of the Executive Committee by providing strategic guidance for BITSS’ continued development, access to…
Win a prize guessing how much trial registration reduces publication bias!
Does trial registration make an impact on publication bias? Knowing the answer could earn you a cash prize! Macartan Humphreys (Columbia, Political Science) and collaborators Albert Fang and Grant Gordon are doing research on how publication (and publication bias) changed after the introduction of registration in clinical trials. They also want you to guess what the changes…
Join an Open Call on Reproducibility Tomorrow at 11 am (ET)
BITSS Project Scientist Garret Christensen will be participating in a discussion with the Mozilla Science Lab on reproducibility in research tomorrow at 11 am ET. The call is open to the public. For those interested in joining, more information can be found here.
Annual Meeting Speakers Now on YouTube!
In case you missed the 2014 BITSS Research Transparency Forum, you can watch the presentations of all five speakers featured on the BITSS YouTube Channel and embedded in our Annual Meeting page. The YouTube channel also includes videos with interviews from the BITSS Board on the importance of research transparency.