Biology subjects

Dadu, A.

Publications and source records attributed to Dadu, A..

6 recordsLinked to original sources

bioRxiv2025

CARDBiomedBench: A Benchmark for Evaluating Large Language Model Performance in Biomedical Research

BackgroundsBiomedical research requires sophisticated understanding and reasoning across multiple specializations. While large language models (LLMs) show promise in scientific applications, their capability to safely and accurately support complex biomedical research remains uncertain. MethodsWe present CARDBiomedBench, a novel question-and-answer benchmark for evaluating LLMs in biomedical research. For our pilot implementation, we focus on neurodegenerative diseases (NDDs), a domain requiring integration of genetic, molecular, and clinical knowledge. The benchmark combines expert-annotated question-answer (Q/A) pairs with semi-automated data augmentation, drawing from authoritative public resources including drug development data, genome-wide association studies (GWAS), and Summary-data based Mendelian Randomization (SMR) analyses. We evaluated seven private and open-source LLMs across ten biological categories and nine reasoning skills, using novel metrics to assess both response quality and safety. ResultsOur benchmark comprises over 68,000 Q/A pairs, enabling robust evaluation of LLM performance. Current state-of-the-art models show significant limitations: models like Claude-3.5-Sonnet demonstrates excessive caution (Response Quality Rate: 25% [95% CI: 25% {+/-} 1], Safety Rate: 76% {+/-} 1), while others like ChatGPT-4o exhibits both poor accuracy and unsafe behavior (Response Quality Rate: 37% {+/-} 1, Safety Rate: 31% {+/-} 1). These findings reveal fundamental gaps in LLMs ability to handle complex biomedical information. ConclusionCARDBiomedBench establishes a rigorous standard for assessing LLM capabilities in biomedical research. Our pilot evaluation in the NDD domain reveals critical limitations in current models ability to safely and accurately process complex scientific information. Future iterations will expand to other biomedical domains, supporting the development of more reliable AI systems for accelerating scientific discovery.

bioinformatics↗

bioRxiv2023

Federated Learning for multi-omics: a performance evaluation in Parkinson's disease

While machine learning (ML) research has recently grown more in popularity, its application in the omics domain is constrained by access to sufficiently large, high-quality datasets needed to train ML models. Federated Learning (FL) represents an opportunity to enable collaborative curation of such datasets among participating institutions. We compare the simulated performance of several models trained using FL against classically trained ML models on the task of multi-omics Parkinsons Disease prediction. We find that FL model performance tracks centrally trained ML models, where the most performant FL model achieves an AUC-PR of 0.876 {+/-} 0.009, 0.014 {+/-} 0.003 less than its centrally trained variation. We also determine that the dispersion of samples within a federation plays a meaningful role in model performance. Our study implements several open source FL frameworks and aims to highlight some of the challenges and opportunities when applying these collaborative methods in multi-omics studies. The Bigger PictureThe wide-scale application of artificial intelligence and computationally intensive analytical approaches in the biomedical and clinical domain is largely restricted by access to sufficient training data. This data scarcity exists due to the isolated nature of biomedical and clinical institutions, mandated by patient privacy policies in the health system or government legislation. Federated Learning (FL), a machine learning approach that facilitates collaborative model training is a promising strategy to address these restrictions. Therefore, understanding the limitations of cooperatively trained FL models, and their performance differences to similar, centrally trained models, is crucial to valuing their implementation in the broader biomedical research community.

bioinformatics↗

bioRxiv2022

Application of Aligned-UMAP to longitudinal biomedical studies

Longitudinal multi-dimensional biological datasets are ubiquitous and highly abundant. These datasets are essential to understanding disease progression, identifying subtypes, and drug discovery. Discovering meaningful patterns or disease pathophysiologies in these datasets is challenging due to their high dimensionality, making it difficult to visualize hidden patterns. Several methods have been developed for dimensionality reduction, but they are limited to cross-sectional datasets. Recently proposed Aligned-UMAP, an extension of the UMAP algorithm, can visualize high-dimensional longitudinal datasets. In this work, we applied Aligned-UMAP on a broad spectrum of clinical, imaging, proteomics, and single-cell datasets. Aligned-UMAP reveals time-dependent hidden patterns when color-coded with the metadata. We found that the algorithm parameters also play a crucial role and must be tuned carefully to utilize the algorithms potential fully. Altogether, based on its ease of use and our evaluation of its performance on different modalities, we anticipate that Aligned-UMAP will be a valuable tool for the biomedical community. We also believe our benchmarking study becomes more important as more and more high-dimensional longitudinal data in biomedical research becomes available. Highlights- explored the utility of Aligned-UMAP in longitudinal biomedical datasets - offer insights on optimal uses for the technique - provide recommendations for best practices In BriefHigh-dimensional longitudinal data is prevalent yet understudied in biological literature. High-dimensional data analysis starts with projecting the data to low dimensions to visualize and understand the underlying data structure. Though few methods are available for visualizing high dimensional longitudinal data, they are not studied extensively in real-world biological datasets. A recently developed nonlinear dimensionality reduction technique, Aligned-UMAP, analyzes sequential data. Here, we give an overview of applications of Aligned-UMAP on various biomedical datasets. We further provide recommendations for best practices and offer insights on optimal uses for the technique.

bioinformatics↗

bioRxiv2022

Identification and prediction of Parkinson's disease subtypes and progression using machine learning in two cohorts.

BackgroundThe clinical manifestations of Parkinsons disease (PD) are characterized by heterogeneity in age at onset, disease duration, rate of progression, and the constellation of motor versus non-motor features. There is an unmet need for the characterization of distinct disease subtypes as well as improved, individualized predictions of the disease course. The emergence of machine learning to detect hidden patterns in complex, multi-dimensional datasets provides unparalleled opportunities to address this critical need. Methods and FindingsWe used unsupervised and supervised machine learning methods on comprehensive, longitudinal clinical data from the Parkinsons Disease Progression Marker Initiative (PPMI) (n = 294 cases) to identify patient subtypes and to predict disease progression. The resulting models were validated in an independent, clinically well-characterized cohort from the Parkinsons Disease Biomarker Program (PDBP) (n = 263 cases). Our analysis distinguished three distinct disease subtypes with highly predictable progression rates, corresponding to slow, moderate, and fast disease progression. We achieved highly accurate projections of disease progression five years after initial diagnosis with an average area under the curve (AUC) of 0.92 (95% CI: 0.95 {+/-} 0.01 for the slower progressing group (PDvec1), 0.87 {+/-} 0.03 for moderate progressors, and 0.95 {+/-} 0.02 for the fast progressing group (PDvec3). We identified serum neurofilament light (Nfl) as a significant indicator of fast disease progression among other key biomarkers of interest. We replicated these findings in an independent validation cohort, released the analytical code, and developed models in an open science manner. ConclusionsOur data-driven study provides insights to deconstruct PD heterogeneity. This approach could have immediate implications for clinical trials by improving the detection of significant clinical outcomes that might have been masked by cohort heterogeneity. We anticipate that machine learning models will improve patient counseling, clinical trial design, allocation of healthcare resources, and ultimately individualized patient care.

bioinformatics↗

bioRxiv2021

A fully automated FAIMS-DIA proteomic pipeline for high-throughput characterization of iPSC-derived neurons

Fully automated proteomic pipelines have the potential to achieve deep coverage of cellular proteomes with high throughput and scalability. However, it is important to evaluate performance, including both reproducibility and ability to provide meaningful levels of biological insight. Here, we present an approach combining high field asymmetric waveform ion mobility spectrometer (FAIMS) interface and data independent acquisition (DIA) proteomics approach developed as part of the induced pluripotent stem cell (iPSC) Neurodegenerative Disease Initiative (iNDI), a large-scale effort to understand how inherited diseases may manifest in neuronal cells. Our FAIMS-DIA approach identified more than 8000 proteins per mass spectrometry (MS) acquisition as well as superior total identification, reproducibility, and accuracy compared to other existing DIA methods. Next, we applied this approach to perform a longitudinal proteomic profiling of the differentiation of iPSC-derived neurons from the KOLF2.1J parental line used in iNDI. This analysis demonstrated a steady increase in expression of mature cortical neuron markers over the course of neuron differentiation. We validated the performance of our proteomics pipeline by comparing it to single cell RNA-Seq datasets obtained in parallel, confirming expression of key markers and cell type annotations. An interactive webapp of this temporal data is available for aligned-UMAP visualization and data browsing (https://share.streamlit.io/anant-droid/singlecellumap). In summary, we report an extensively optimized and validated proteomic pipeline that will be suitable for large-scale studies such as iNDI.

neuroscience↗

bioRxiv2021

Multi-Modality Machine Learning Predicting Parkinsons Disease

BackgroundPersonalized medicine promises individualized disease prediction and treatment. The convergence of machine learning (ML) and available multi-modal data is key moving forward. We build upon previous work to deliver multi-modal predictions of Parkinsons Disease (PD). MethodsWe performed automated ML on multi-modal data from the Parkinsons Progression Marker Initiative (PPMI). After selecting the best performing algorithm, all PPMI data was used to tune the selected model. The model was validated in the Parkinsons Disease Biomarker Program (PDBP) dataset. Finally, networks were built to identify gene communities specific to PD. FindingsOur initial model showed an area under the curve (AUC) of 89.72% for the diagnosis of PD. The tuned model was then tested for validation on external data (PDBP, AUC 85.03%). Optimizing thresholds for classification, increased the diagnosis prediction accuracy (balanced accuracy) and other metrics. Combining data modalities outperforms the single biomarker paradigm. UPSIT was the largest contributing predictor for the classification of PD. The transcriptomic data was used to construct a network of disease-relevant transcripts. InterpretationWe have built a model using an automated ML pipeline to make improved multi-omic predictions of PD. The model developed improves disease risk prediction, a critical step for better assessment of PD risk. We constructed gene expression networks for the next generation of genomics-derived interventions. Our automated ML approach allows complex predictive models to be reproducible and accessible to the community. FundingNational Institute on Aging, National Institute of Neurological Disorders and Stroke, the Michael J. Fox Foundation, and the Global Parkinsons Genetics Program. RESEARCH IN CONTEXTO_ST_ABSEvidence before this studyC_ST_ABSPrior research into predictors of Parkinsons disease (PD) has either used basic statistical methods to make predictions across data modalities, or they have focused on a single data type or biomarker model. We have done this using an open-source automated machine learning (ML) framework on extensive multi-modal data, which we believe yields robust and reproducible results. We consider this the first true multi-modality ML study of PD risk classification. Added value of this studyWe used a variety of linear, non-linear, kernel, neural networks, and ensemble ML algorithms to generate an accurate classification of both cases and controls in independent datasets using data that is not involved in PD diagnosis itself at study recruitment. The model built in this paper significantly improves upon our previous models that used the entire training dataset in previous work1. Building on this earlier work, we showed that the PD diagnosis can be refined using improved algorithmic classification tools that may yield potential biological insights. We have taken careful consideration to develop and validate this model using public controlled-access datasets and an open-source ML framework to allow for reproducible and transparent results. Implications of all available evidenceTraining, validating, and tuning a diagnostic algorithm for PD will allow us to augment clinical diagnoses or risk assessments with less need for complex and expensive exams. Going forward, these models can be built on remote or asynchronously collected data which may be important in a growing telemedicine paradigm. More refined diagnostics will also increase clinical trial efficiency by potentially refining phenotyping and predicting onset, allowing providers to identify potential cases earlier. Early detection could lead to improved treatment response and higher efficacy. Finally, as part of our workflow, we built new networks representing communities of genes correlated in PD cases in a hypothesis-free manner, showing how new and existing genes may be connected and highlighting therapeutic opportunities.

genomics↗