Tracking claim changes from preprint to publication across 72,644 biomedical studies using large language models

Abstract

Preprints now disseminate a large share of biomedical research before peer review. Because they have not yet passed peer review, some scientists regard preprint claims as unverified or potentially unreliable, yet how much those claims change before publication has so far been quantified only in smaller cohorts, with results that vary by field and topic. Here, we compiled every bioRxiv preprint posted between 2018 and 2025 that we could match by DOI to a peer-reviewed published version, yielding 72,644 preprint-publication pairs. Using a large language model (Claude Sonnet 4.6), we parsed every preprint-publication abstract pair into one primary and two secondary claims, and classified each pair for content change (unchanged, minor, major) and hedging shift (more cautious, more confident, unchanged). On a validation subsample, the model agreed with two independent domain experts about as well as the experts agreed with each other (Cohens kappa 0.63 to 0.66). The primary claim was unchanged in 39.9% of abstracts, minorly revised in 50.0%, and substantially revised in only 10.2%. Hedging shifts were uncommon and asymmetric, with twice as many claims becoming more cautious as more confident (8.4% vs 4.2%). Major revisions were more frequent after long peer review (14.1% in the slowest versus 7.0% in the fastest tertile of review time) and declined over the study period (17.0% in 2019 to 5.7% in 2024). Over the same period, biomedical papers that were never posted as preprints were retracted at roughly twice the rate of those that were. Together, these data show that the move from preprint to peer-reviewed publication leaves the central claims of most biomedical abstracts intact, indicating that preprints are a reliable source of biomedical research.