This interesting paper from social scientist Ryan Briggs and colleagues attempts to quantify the widely believed bias against the publication of null results using a dataset of 100,000 articles published in political science journals. Here is a snippet of the abstract (emphasis mine):
In this article, we use large language models to extract granular and validated data on about 100,000 articles published in over 150 political science journals from 2010 to 2024. We show that fewer than 2% of articles that rely on statistical methods report null-only findings in their abstracts, while over 90% of papers highlight significant results. To put these findings in perspective, we develop and calibrate a simple model of publication bias. Across a range of plausible assumptions, we find that statistically significant results are estimated to be one to two orders of magnitude more likely to enter the published record than null results. Leveraging metadata extracted from individual articles, we show that the pattern of strong SoS holds across sub-fields, journals, methods, and time periods. However, a few factors such as pre-registration and randomized experiments correlate with greater acceptance of null results.
A depressing story for science but maybe a feel-good story for meta-science?