–Garret Christensen, BITSS Project Scientist
Last month a colleague of mine from the Berkeley Institute for Data Science (BIDS), John Bohannon, and I organized a panel discussion on the subject of reproducible journalism for Stanford’s Computation + Journalism Symposium. John is a contributing correspondent for Science, but he’s been visiting BIDS for a while, working to develop the concept of reproducible journalism. This is the question of how data journalism can be reproducible–how should journalists deal with the question of anonymous sources, or leaked data, etc.? Is there a difference between reproducible journalism and reproducible science? What formats or platforms should journalists use to share their data? What’s the analog of academic peer review for journalism stories?
We also dealt with the related question of how journalists can cover reproducibility of scientific research itself as a topic. How should journalists deal with subject areas that have had reproducibility issues? Since statistics isn’t uncertainty, how should journalists accurately relay the uncertainty without portraying science as fundamentally broken?
Our panel of experts consisted of:
- Christie Aschwanden, Lead Science Writer at FiveThirtyEight
- Peter Aldhous, Science and Health Reporter at BuzzFeed News
- Daniele Fanelli, Senior Research Scientist at Stanford’s METRICS
- Simine Vazire, Associate Professor of Psychology at UC Davis
- Dan Nguyen, Stanford University Hearst Professional-in-Residence
I don’t know that we exactly answered all of those questions, nor did we solve the reproducibility crisis, but there were several interesting takeaways and ideas proposed. Among others:
- Peter Aldhous was pleasantly surprised at how many journalists in the audience had publicly shared the data from their articles.
- Christie Aschwanden said that FiveThirtyEight had made an active decision to not write news stories on single studies–instead opting for articles summarizing literatures, akin to meta-analysis.
- Simine Vazire suggested that scientists and journalists could both benefit from presenting the most damning piece of evidence (i.e., the piece most contrary to the main claim of the article) to more accurately portray uncertainty.
- Daniele Fanelli suggested that journalists could make transcripts of interviews available so interested readers could see the “raw data” behind the conclusions of the journalist, but others on the panel thought this might not fly with interview subjects, or would change the willingness of interview subjects to speak freely.
You can watch the entire panel below: