Preprints in motion: tracking changes between posting and journal publication

Polka, J. K.; Dey, G.; Palfy, M.; Nanni, F.; Brierley, L.; Fraser, N.; Coates, J. A.

doi:10.1101/2021.02.20.432090

Home/Archive/bioRxiv

bioRxiv · 10.1101/2021.02.20.432090

Preprints in motion: tracking changes between posting and journal publication

Polka, J. K.·Dey, G.·Palfy, M.·Nanni, F.·Brierley, L.·Fraser, N.·Coates, J. A.

View original source ↗Export metadata ↓

Abstract

Amidst the COVID-19 pandemic, preprints in the biomedical sciences are being posted and accessed at unprecedented rates, drawing widespread attention from the general public, press and policymakers for the first time. This phenomenon has sharpened longstanding questions about the reliability of information shared prior to journal peer review. Does the information shared in preprints typically withstand the scrutiny of peer review, or are conclusions likely to change in the version of record? We assessed preprints from bioRxiv and medRxiv that had been posted and subsequently published in a journal through 30th April 2020, representing the initial phase of the pandemic response. We utilised a combination of automatic and manual annotations to quantify how an article changed between the preprinted and published version. We found that the total number of figure panels and tables changed little between preprint and published articles. Moreover, the conclusions of 7.2% of non-COVID-19-related and 17.2% of COVID-19-related abstracts undergo a discrete change by the time of publication, but the majority of these changes do not qualitatively change the conclusions of the paper.

Explore related subjects

scientific communication and education Preprint

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Polka, J. K., Dey, G., Palfy, M., Nanni, F., Brierley, L., Fraser, N., Coates, J. A.. 2021-02-20. Preprints in motion: tracking changes between posting and journal publication. https://doi.org/10.1101/2021.02.20.432090

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗

bioRxiv2021

Inferring livestock movement networks from archived data to support infectious disease control in developing countries

The use of network analysis to support livestock disease control in low middle-income countries (LMICs) has historically been hampered by the cost of generating empirical data in the absence of animal movement recording schemes. To fill this gap, methods which exploit freely available demographic and archived molecular data can be used to generate livestock networks based on gravity and phylogeographic modelling techniques, respectively. However, questions remain on the performance of these methods in capturing the topology of empirical networks. Here, we compare output from these network methodologies to a network constructed from either empirical data or randomly generated data. To facilitate this comparison, the spread of infectious diseases was simulated, it is this evaluation that demonstrates their potential utility to inform robust livestock disease control strategies. The molecular network was the closest approximation to the empirical network, both in relation to topological and epidemic characteristics, whereas size of epidemics in the gravity network tended to be larger, better agreement across all three networks was observed when; a) total nodes infected, b) percentage infection take off were compared. These methods consistently identified the same important animal movement and trade hotspots as the empirical networks. We therefore consider this proof-of-concept that demographic data such as censuses and archived molecular data could be repurposed to inform livestock disease management in LMICs. Author summaryLive animal movements in Africa represent a significant risk of transmission and spread of infectious diseases in livestock populations, and therefore, have direct implications on the food security of the continent. Here we explore the potential utility of available data to support control strategies, by comparing movement networks inferred from such data i.e. census and pathogen molecular data using gravity modelling and phylogeography respectively. Their utility is evaluated by comparing their topology and disease spread characteristics to empirical live animal movement. Based on our results, we posit that archived data can be repurposed to support infectious disease control on the African continent.

scientific communication and education↗

bioRxiv2021

scite: a smart citation index that displays the context of citations and classifies their intent using deep learning

Citation indices are tools used by the academic community for research and research evaluation which aggregate scientific literature output and measure scientific impact by collating citation counts. Citation indices help measure the interconnections between scientific papers but fall short because they only display paper titles, authors, and the date of publications, and fail to communicate contextual information about why a citation was made. The usage of citations in research evaluation without due consideration to context can be problematic, if only because a citation that disputes a paper is treated the same as a citation that supports it. To solve this problem, we have used machine learning and other techniques to develop a "smart citation index" called scite, which categorizes citations based on context. Scite shows how a citation was used by displaying the surrounding textual context from the citing paper, and a classification from our deep learning model that indicates whether the statement provides supporting or disputing evidence for a referenced work, or simply mentions it. Scite has been developed by analyzing over 23 million full-text scientific articles and currently has a database of more than 800 million classified citation statements. Here we describe how scite works and how it can be used to further research and research evaluation.

scientific communication and education↗