OSF

Assessing Bias from the (Mis)Use of Covariates: A Meta-Analysis Political ScienceSSMART

Gabriel Lenz Alexander Sahn

How often do articles depend on suppression effects for their findings? How often do they disclose this fact? By suppression effects, we mean covariate-induced increases in effect sizes. Researchers generally scrutinize suppression effects as they want reassurance that researchers have a strong explanation for effect size increases, especially when the statistical significance of the key finding depends on them. In this article, we find that 30-40% of observational articles in a leading journal depend on suppression effects for statistical significance. Although suppression effects are of course potentially justifiable — to address suppressor variables — none of the articles justifies or discloses them. These findings may point to a hole in the review process: journals are accepting articles that depend on suppression effects without readers, reviewers, or editors being made aware.

Find the most recent version of this paper here.