Guest post by Jamie Monogan (University of Georgia) A conversation is emerging in the social sciences over the merits of study registration and whether it should be the next step we take in raising research transparency. The notion of study registration is that, prior to observing outcome data, a researcher can publicly…
Call for Papers: Special Issue of Comparative Political Studies on Research Transparency
From Michael Findley (University of Texas), Nathan Jensen (Washington University), Edmund Malesky (Duke), and Thomas Pepinsky (Cornell) We invite proposals for a special issue of Comparative Political Studies (CPS) on research transparency in the social sciences. Proposals for original research papers using quantitative or qualitative approaches, and collecting quantitative or qualitative data…
Scientific Misconduct in the Middle East
On plagiarism and fraud in the Middle Eastern research community (by Ranya Stamboliyska): Gallons of digital ink have been spilt discussing depressing laundry lists of misconduct cases in the west (and more recently, in China). There is, however, very little on unethical behaviour in the Arab world, despite the wide number of Mid-Eastern students…
Tonight! Data Science Meets Social Science
Tonight in Berkeley! June 5, 2014 6:00pm — 7:30pm. David Brower Center (Kinzie Room), 2150 Allston Way. All across the social sciences we can see a convergence around the ideals of openness and reproducibility. Over the past years, the injection of ways of thinking and working from scientific computing into social science research has helped…
BITSS Summer Institute – Summary of the First Day
BITSS is currently holding its first summer institute in transparency practices for empirical research. The meeting is taking place in Berkeley, CA with 30+ graduate students and junior faculty in the attendance. Ted Miguel (Economics, UC Berkeley), one of the founding members of BITSS, started with an overview of conceptual issues in…
What can be done to prevent the proliferation of errors in academic publications?
Every now and again a paper is published on the number of errors made in academic articles. These papers document the frequency of conceptual errors, factual errors, errors in abstracts, errors in quotations, and errors in reference lists. James Hartley reports that the data are alarming, but suggests a possible way of…
Replicate it! A Proposal to Foster Knowledge Accumulation
Thad Dunning and Susan D. Hyde in the Washington Post: Like many social scientists, we take it almost as an article of faith that scientific methods will advance our knowledge about how the world works. The growing use by social scientists of strong research designs — for example, randomized controlled experiments or…
Is public criticism of science bullying?
The Ethics of Using and Sharing Data for Development Programming (5/22 @ NYC)
The Responsible Data Forum is a new effort to map the ethical, legal, privacy and security challenges surrounding the increased use and sharing of data in development programming. The Forum aims to explore the ways in which these challenges are experienced in project design and implementation, as well as when project data is…
Flawed Research On Your Plate
You might want to reconsider paying extra dollar for these fish oil supplements. A new study said most of the research literature on the cardiovascular benefits of omega-3 fatty acids is flawed. In the early 1970s, two Danish researchers started to investigate the diet of Greenland’s Inuit populations, which were believed to live longer than their Caucasian counterparts. The study…
Git/GitHub: A Primer for Researchers
Tomorrow — West Coast Experiments Conference @ Claremont Graduate University
Tomorrow at Claremont Graduate University (Burkle Building, Room 16). One workshop to focus on research transparency and integrity: Session II (12:45 to 3:15, 45 minutes for the papers, one hour for the training session) Neil Malhotra (Stanford), “Publication Bias in Political Science: Using TESS Experiments to Unlock the File Drawer” Thad Dunning (Berkeley),…
The Reformation: Can Social Scientists Save Themselves?
From Jerry Adler in the Pacific Standard—on the credibility crisis in social science research, publication bias, data manipulation, and non-replicability. Featuring BITSS aficionados Brian Nosek, Joe Simmons, Uri Simonsohn and Leif Nelson. Something unprecedented has occurred in the last couple of decades in the social sciences. Overlaid on the usual academic incentives of…
Open Data Workshops (London, UK)
The Open Data Institute is organizing multiple workshops (May 12-16, July 21-25 | London, UK) to introduce the technical, commercial and legal aspects of open data. The courses aim to highlight key opportunities for working with open data and how these can be exploited across government, business and society. The workshops are designed to enable…
Reproducibility Session @ AAAS Forum on Science and Technology Policy
A core value of science is the ability to reproduce the findings of others in order to check for methodological rigor, errors, plausible interpretations, and/or misconduct. However, for a variety of reasons, from lack of time or resources to little professional recognition or credit for doing so, science may be failing to…
New book: Implementing Reproducible Research
New book from Victoria Stodden, Friedrich Leisch, and Roger D. Peng: “Implementing Reproducible Research“. In many of today’s research fields, including biomedicine, computational tools are increasingly being used so that the results can be reproduced. Researchers are now encouraged to incorporate software, data, and code in their academic papers so that others can…
Just published: SAGE handbook on Managing and Sharing Research Data
Research funders in the UK, USA and across Europe are implementing data management and sharing policies to maximize openness of data, transparency and accountability of the research they support. Written by experts from the UK Data Archive with over 20 years experience, this book gives post-graduate students, researchers and research support staff…
Feedback Wanted: Publishers & Data Access
Panel on Transparency and Replication @ EGAP 11 (Berkeley, CA — Friday 4/11)
Experiments in Governance and Politics (EGAP) will be holding its eleventh bi-annual meeting in Berkeley, CA this Friday and Saturday (April 11-12). In addition to research design workshops and recent papers presentations, the meeting will feature an interdisciplinary panel on transparency and replication (Friday, 3.50-6.00pm): 3:50 – 4:10 PM Thad Dunning “EGAP Regranting…
Reproducibility Pioneers: Political Scientists Assigning Replications to Students
Transparency Seminar @ NYU
Ted Miguel (UC Berkeley) will provide an overview of and update on BITSS at New York University on Friday, April 4. Sponsored by the Development Research Institute, the presentation will cover recent progress toward research transparency in the social sciences, and make the case for standards and practices that help realign scholarly incentives with scholarly values. Time: Friday,…
Deadline Extended for BITSS Summer Institute!
The deadline for applying to the BITSS Summer Institute (June 2-5, 2014 – Berkeley, CA) has been extended to Wednesday, April 2. The week-long course will cover a broad array of topics related to openness and transparency in social science research, including the theory and implementation of pre-analysis plans; building reproducible knowledge; emerging techniques to carry…
A Discussion with Victoria Stodden (April 3 — Washington, DC)
The AAAS FIRE and Big Data Affinity groups are co-hosting an informal discussion with Victoria Stodden (Columbia University, Department of Statistics) on Thursday April 3 at 6pm in the Kogod Courtyard of the National Portrait Gallery, in Washington, DC. The debate will focus on the role of data and software in scientific research and funding.…
Replication Panels at ISA 2014
The 55th Annual Convention of the International Studies Association (March 26-29, 2014 – Toronto, Canada) will feature two panels on replication and reproducibility: Friday, March 28 8:15 AM – 10:00 AM Replication in International Relations: How Journal Data Policies and Replication in Teaching Can Improve Reproducibility Standards Friday, March 28 10:30 AM…
20 Research Transparency Initiatives to Follow on Twitter
A non-exhaustive list of groups and initiatives with regular postings on research integrity, transparency, and reproducibility: CA Digital Library @CalDigLib CenterForOpenScience @OSFramework Dataverse Network @thedataorg Dexy @dexyit D-Lab @DLabAtBerkeley DuraSpace Community @duraspace figshare @figshare Force11.org @force11rescomm GitHub @github ICPSR @ICPSR
Deadline Coming Up for BITSS Summer Institute!
Applications for the first BITSS Summer Institute (June 2-5, 2014 – Berkeley, CA) are due by Wednesday, March 26. The workshop will inform participants about the latest trends in the shift towards increased transparency in the social sciences, providing an overview of new tools and techniques for rigorous research in economics, political science, psychology,…
Git can facilitate greater reproducibility and increased transparency in science
A well-articulated piece on why Git is a useful too for open, reproducible science (by Karthik Ram): Reproducibility is the hallmark of good science. Maintaining a high degree of transparency in scientific reporting is essential not just for gaining trust and credibility within the scientific community but also for facilitating the development of new ideas.…
Endorse the Joint Declaration of Data Citation Principles
Sound, reproducible scholarship rests upon a foundation of robust, accessible data. For this to be so in practice as well as theory, data must be accorded due importance in the practice of scholarship and in the enduring scholarly record. In other words, data should be considered legitimate, citable products of research. Data…
“We shouldn’t tell people to do whatever their moms tell them to do”
Kevin Esterling, Perspective from Political Science (II) This is the fifth post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
“Interdisciplinary initiatives are where most progress happens and where exciting innovations come from”
Maya Petersen, Perspective from Medicine This is the fourth post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
Funding Opportunity for Coordinated Research on Political Accountability
The Experiments in Governance and Politics network (EGAP) is requesting statements of interest for leading edge experimental research projects on political accountability in developing countries. This grant round is specifically designed to foster knowledge cumulation across studies. Successful applicants will engage in closely related projects and adhere to a common set of…
Announcing the BITSS Summer Institute
Registration is now open for the first BITSS summer institute (June 2-5, 2014 – Berkeley, CA). The workshop is designed for students, post-docs, and junior faculty eager to learn more about new transparency tools available and appropriate for their own research in economics, political science, psychology or another related discipline. Course description, list of…
“Creating tools is not the biggest challenge. The biggest challenge is getting people to use them”
Brian Nosek, Perspective from Psychology This is the third post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
Scientific Pride and Prejudice
Or why Jane Austen might well be the first game theorist. Science is in crisis, just when we need it most […] A major root of the crisis is selective use of data. Scientists, eager to make striking new claims, focus only on evidence that supports their preconceptions. Psychologists call this “confirmation bias”:…
“I would like to see us demonstrating by example the value of this form of science”
Don Green, Perspective from Political Science This is the second post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
Twenty Tips For Interpreting Scientific Claims
A useful list of 20 concepts to help decision-makers parse how evidence can contribute to a decision, and potentially avoid undue influence by those with vested interests. Calls for the closer integration of science in political decision-making have been commonplace for decades. However, there are serious problems in the application of…
The changing face of psychology
Important changes are underway in psychology. Transparency, reliability, and adherence to scientific methods are the key words for 2014, says a recent article in The Guardian. A growing number of psychologists – particularly the younger generation – are fed up with results that don’t replicate, journals that value story-telling over truth, and an…
“There is an enormous amount of work, but we do not have the right incentives to do it”
Rachel Glennerster, Perspective from Economics This is the first post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. This interview was recorded on December 12, 2013 at the University of California, Berkeley.
Science Magazine Embarks on Reproducibility Quest
Marcia McNutt, Editor-in-Chief of Science, in the January 17 editorial: Science advances on a foundation of trusted discoveries. Reproducing an experiment is one important approach that scientists use to gain confidence in their conclusions. Recently, the scientific community was shaken by reports that a troubling proportion of peer-reviewed preclinical studies are not…
BITSS is hiring!
Interested in improving the standards of rigor in empirical scientific research? Eager to collaborate with leading social science researchers to promote research transparency? Wishing to stay abreast of new advances in empirical research methods and transparency software development? BITSS is looking for a Program Associate to support the initiative’s evaluation and outreach…
Best Practices for Scientific Computing
A set of best practices for scientific computing, by Wilson et al (2014). Scientists spend an increasing amount of time building and using software. However, most scientists are never taught how to do this efficiently. As a result, many are unaware of tools and practices that would allow them to write more…
When is an error not an error?
Guest post by Annette N. Brown and Benjamin D. K. Wood on the World Bank Development Impact blog: We are seeing a similar propensity for replication researchers to use the word “error” (or “mistake” or “wrong”) and for this language to cause contentious discussions between the original authors and replication researchers. The…
Replication in Political Science
Here is a new initiative seeking to promote replications of quantitative work in political science. The group of researchers behind this project aims to create a site that will publish and organize replications done by graduate students in their courses — by which they mean the exercise of conducting re-analyses using the original data/code, as well…
Preregistration: Not just for the Empiro-zealots
Leif Nelson making the case for pre-registration: I recently joined a large group of academics in co-authoring a paper looking at how political science, economics, and psychology are working to increase transparency in scientific publications. Psychology is leading, by the way. Working on that paper (and the figure below) actually changed my…
AEA 2014: The Use of Pre-analysis Plans and Other Transparency Approaches in Economics
By Guillaume Kroll (CEGA) BITSS held a session on research transparency at the 2014 Annual Meeting of the American Economic Association on January 5 in Philadelphia. UC Berkeley economist Ted Miguel, who was presiding over the session, kicked off the discussion by highlighting the growing interest in transparency across social science disciplines.…
Replication & Reproducibility 2013: The best stories
BITSS Affiliates Advocate for Higher Transparency Standards in Science Magazine
In the January 3, 2014 edition of Science Magazine, an interdisciplinary group of 19 BITSS affiliates reviews recent efforts to promote transparency in the social sciences and make the case for more stringent norms and practices to help boost the quality and credibility of research findings. The authors, led by UC Berkeley…
Doing the right thing: Yale psychology lab retracts monkey papers for inaccurate coding
Do randomized controlled trials engage in less specification searching?
An excerpt from on-going work based on AidGrade’s database of impact evaluation results in development economics. See how RCTs compare with non-RCTs, and how economics compare with political science or sociology. Read more…
Annual Meeting Pictures
Pictures of the BITSS annual conference are up on the event page. Meeting resources coming soon.