Meta-Analytic Sensitivity to Publication Bias (Daniel Quintana LinkedIn Post)

Jan 1, 1 · 2 min read

In this post, Daniel Quintana addresses the common pitfalls in how researchers handle publication bias in meta-analyses, arguing for a more sophisticated, multi-faceted approach to sensitivity analysis.

Key Concepts and Findings:

  • Small-Study Bias vs. Publication Bias: Quintana cautions that standard tools—like Egger’s test and funnel plots—are often misinterpreted. They primarily detect small-study bias (where smaller studies tend to show larger effect sizes) rather than directly measuring publication bias itself.
  • The Need for Direct Correction: He advocates for moving beyond simple visualization and toward methods designed to directly correct for publication bias, such as:
    • PET-PEESE (Precision-Effect Test and Precision-Effect Estimate with Standard Errors)
    • Limit meta-analysis
    • p-uniform
  • The Problem of Assumptions: These correction methods are not ‘one-size-fits-all.’ Because each method relies on unique assumptions about why small studies report larger effects, they can produce significantly different results.
  • Case Study (Oxytocin Research): In an analysis of 530 effect sizes across 185 studies on oxytocin administration, Quintana demonstrated this sensitivity: the estimated mean effect size varied substantially, ranging from d = 0.05 to d = 0.17, depending on the model chosen.
  • Core Recommendation: Meta-analysts should not rely on a single correction method. Instead, they should report results from a range of publication bias methods to demonstrate the sensitivity and robustness of their findings.

Quintana also emphasizes that while these newer methods are becoming more accessible (e.g., through software like JASP), the responsibility lies with the researcher to understand the underlying assumptions of the models they employ.