Uri Simonsohn on replications: It is common for researchers running replications to set their sample size assuming the effect size the original researchers got is correct. So if the original study found an effect-size of d=.73, the replicator assumes the true effect is d=.73, and sets sample size so as to have…
Too Much Trusting, Not Enough Verifying
This week in The Economist: Too many of the findings that fill the academic ether are the result of shoddy experiments or poor analysis […] One reason is the competitiveness of science […] The obligation to “publish or perish” has come to rule over academic life. Competition for jobs is cut-throat […] Nowadays verification (the…
The Imperative to Share Complete Replication Files
“Good research involves publishing complete replication files, making every step of research as explicit and reproducible as is practical.” This is the conclusion from a new paper by political scientist Allan Dafoe (Yale University). Dafoe examines the availability of replication data in political science journals, and concludes that “for the majority of published statistical analyses, […]…
Research Transparency in the Natural Sciences: What can we learn?
By Temina Madon (CEGA, UC Berkeley) As we all know, experimentation in the natural sciences far predates the use of randomized, controlled trials (RCTs) in medicine and the social sciences; some of the earliest controlled experiments were conducted in the 1920s by RA Fisher, an agricultural scientist evaluating new crop varieties across…