bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/726505

Evaluation of Reproducible and Transparent Research Practices in Pulmonology

Smith, C.·Nolan, J.·Tritz, D.·Heavener, T.·Pelton, J.·Cook, K.·Vassar, M.
View original source ↗Export metadata ↓

Abstract

RationaleStudy reproducibility is valuable for validating or refuting results. Provision of reproducibility indicators, such as materials, protocols, and raw data in a study to improve its potential for reproduction. Efforts to reproduce noteworthy studies in the biomedical sciences have resulted in an overwhelming majority of them being found to be unreplicable, causing concern for the integrity of research in other fields, including medical specialities.\n\nObjectiveHere, we analyzed the reproducibility of studies in the field of pulmonology.\n\nMethods300 pulmonology articles were randomly selected from an initial PubMed search for data extraction. Two authors scoured these articles for reproducibility indicators including materials, protocols, raw data, analysis scripts, inclusion in systematic reviews, and citations by replication studies as well as other factors of research transparency including open accessibility, funding source and competing interest disclosures, and study preregistration.\n\nMain ResultsFew publications included statements regarding materials (11%), protocols (1%), data (21%), and analysis script (0%) availability. Less than 10% indicated preregistration. More than half of the publications analyzed failed to provide a funding statement. Conversely, 66% of the publications were open access and 70% included a conflict of interest statement.\n\nConclusionOverall, our study indicates pulmonology research is currently lacking in efforts to increase replicability. Future studies should focus on providing sufficient information regarding materials, protocols, raw data, and analysis scripts, among other indicators, for the sake of clinical decisions that depend on replicable or refutable results from the primary literature.

Explore related subjects

scientific communication and educationPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Smith, C., Nolan, J., Tritz, D., Heavener, T., Pelton, J., Cook, K., Vassar, M.. 2019-08-09. Evaluation of Reproducible and Transparent Research Practices in Pulmonology. https://doi.org/10.1101/726505

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗
bioRxiv2019

PlotTwist - a web app for plotting and annotating time-series data

The results from time-dependent experiments are often used to generate plots that visualize how the data evolves over time. To simplify state-of-the-art data visualization and annotation of data from such experiments, an open source tool was created with R/shiny that does not require coding skills to operate. The freely available web app accepts wide (spreadsheet) and tidy data and offers a range of options to normalize the data. The data from individual objects can be shown in three different ways: (i) lines with unique colors, (ii) small multiples and (iii) heatmap-style display. Next to this, the mean can be displayed with a 95% confidence interval for the visual comparison of different conditions. Several color blind friendly palettes are available to label the data and/or statistics. The plots can be annotated with graphical features and/or text to indicate any perturbations that were applied during the time-lapse experiments. All user-defined settings can be stored for reproducibility of the data visualization. The app is dubbed PlotTwist and is available online: https://huygens.science.uva.nl/PlotTwist\n\n\n\nO_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=166 SRC=\"FIGDIR/small/745612v2_ufig1.gif\" ALT=\"Figure 1\">\nView larger version (34K):\norg.highwire.dtl.DTLVardef@1bad5e6org.highwire.dtl.DTLVardef@1312023org.highwire.dtl.DTLVardef@350cb5org.highwire.dtl.DTLVardef@d556c6_HPS_FORMAT_FIGEXP M_FIG C_FIG

scientific communication and education↗
bioRxiv2019

The Five-Primer Challenge: An inquiry-based laboratory module for synthetic biology

New technologies in DNA synthesis and assembly give genetic engineers complete freedom in genetic design, where virtually any plasmid DNA sequence can be created efficiently and economically. Learning how to design, construct, and test new DNA sequences is a critical skill for researchers in molecular biology and biotechnology. Here we present a student-centered, inquiry-based module in which students learn how to control bacterial gene expression by appplying various DNA assembly techniques. The central activity in this learning module is termed the Five-Primer Challenge. Each student is allowed to order up to five 60-mer oligonucleotide primers to then modify a GFP expression plasmid with the goal of increasing GFP expression as much as possible. This module was developed and implemented at the 2016 Cold Spring Harbor Laboratory Synthetic Biology Course, and was effective at engaging students in critical thinking and in promoting student learning.

scientific communication and education↗