bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2020.09.18.303701

Reporting quality of trend analyses in leading medicine and oncology journals

Yuan, X.·Lin, Y.·Wang, Y.·Zhang, L.
View original source ↗Export metadata ↓

Abstract

Reporting quality of clinical research is critical for evidence-based medicine and reproducibility of clinical research. Most of the works focused on reporting quality of clinical trials and observational longitudinal studies. However, few focused on that of trend analyses. The reporting of recommended statistic metrics in trend analyses was also largely unclear. Therefore, we examined reporting quality of the trend analyses based on reporting of recommended statistic metrics. We systemically searched the PubMed for the trend-analysis articles published in 10 leading medicine and oncology journals during the 11 years from 2008 to 2018. The studies published after 2019 were not included due to the sudden, significant increase of publication number during and immediately after the COVID-19 pandemic. Only original articles, research letters and meta-analyses/systematic reviews were included. We scored the reporting quality of these articles based on whether they reported p-values/effect-sizes, and beta/co-efficient/slope/annual-percentage-change (APC). There were 297 qualified articles, among which 193 (66.0%) and 216 (72.7%) articles reported P-value and effect-size, respectively. Only 13 (5.8%) analyses reported neither p-value/effect size nor beta/coefficient/slope/APC. In multivariable regression models, author affiliation of epidemiology department was associated with less reporting effect-size, but that of statistics department with more reporting. Interestingly, U.S. senior-authors (versus non-U.S.) more likely reported p-values. No factors were independently linked to reporting APC. The reporting quality of trend analyses in leading medicine and oncology journals appear moderate and should be further improved. We thus call for more research and awareness of reporting-quality in trend analyses in oncology research and beyond.

Explore related subjects

scientific communication and educationPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yuan, X., Lin, Y., Wang, Y., Zhang, L.. 2020-09-21. Reporting quality of trend analyses in leading medicine and oncology journals. https://doi.org/10.1101/2020.09.18.303701

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗
bioRxiv2022

Bridging the gap between formal theory and scientific reform practices

A core problem that has been addressed in the scientific reform movement so far is the low rates of reproducibility of research results. Mainstream reform literature aimed at increasing reproducibility rates by implementing changes in research practice and scientific policy. At the sidelines of reform, theoreticians have worked on understanding the underlying causes of irreproducibility from ground up. Each approach faces its own challenges. While mainstream focus on swift practical changes has not been buttressed by sound theoretical arguments, theoretical work is slow and initially is only capable of answering questions in idealized setups, removed from real life constraints. In this article, we continue to develop theoretical foundations in understanding non-exact replications and meta-hypothesis tests in multi-lab replication studies, juxtapose these theoretical intuitions with practical reform examples, and expose challenges we face. In our estimation, a major challenge in the next generation of the reform movement is to bridge the gap between theoretical knowledge and practical advancements.

scientific communication and education↗
bioRxiv2022

Taxonomy of interventions at academic institutions to improve research quality

Research institutions and researchers have become increasingly concerned about poor research reproducibility and replicability, and research waste more broadly. Research institutions play an important role and understanding their intervention options is important. This review aims to identify and classify possible interventions to improve research quality, reduce waste, and improve reproducibility and replicability within research-performing institutions. Taxonomy development steps: 1) use of an exemplar paper of journal-level research quality improvement interventions, 2) 2-stage search in PubMed using seed and exemplar articles, and forward and backward citation searching to identify articles evaluating or describing research quality improvement, 3) elicited draft taxonomy feedback from researchers at an open-sciences conference workshop, and 4) cycles of revisions from the research team. The search identified 11 peer-reviewed articles on relevant interventions. Overall, 93 interventions were identified from peer-review literature and researcher reporting. Interventions covered before, during, and after study conduct research stages and whole of institution. Types of intervention included: Tools, Education & Training, Incentives, Modelling & Mentoring, Review & Feedback, Expert involvement, and Policies & Procedures. Identified areas for research institutions to focus on to improve research quality and for further research includes improving incentives to implement quality research practices, evaluating current interventions, encourage no- or low-cost/high-benefit interventions, examine institution research culture, and encourage mentor-mentee relationships.

scientific communication and education↗