Biology subjects

Dhinagar, N. J.

Publications and source records attributed to Dhinagar, N. J..

7 recordsLinked to original sources

bioRxiv2026

CALM-VLM: CALIBRATION AND SELECTIVE PREDICTION IN VISION-LANGUAGE MODELS FOR RELIABLE BRAIN MRI CLASSIFICATION

AO_SCPLOWBSTRACTC_SCPLOWRecent advances in vision-language models (VLMs) have demonstrated strong multimodal capabilities for medical image analysis. However, their confidence in diagnostic predictions is often unclear, limiting adoption in clinical settings. We introduce CALM-VLM (CALibration Mechanism for Vision-Language Models), which integrates confidence calibration and selective prediction into a generative 3D VLM. To create CALM-VLM, we fine-tuned the Med3DVLM architecture for Alzheimers disease (AD) and stroke classification as initial test cases. To improve reliability, we incorporated temperature scaling on the VLMs generative outputs. The calibrated model then selectively abstained from predictions when uncertain; this also improved its diagnostic accuracy. Experiments across multi-site MRI datasets, from 10 countries worldwide, show that CALM-VLM improved confidence relative to uncalibrated VLMs. Coverage-adjusted test receiver-operator characteristic curve-area under the curve (ROC-AUC) increased by 5% to 13% for both diagnostic tasks across independent test sets. Our calibrated VLM achieved a test ROC-AUC of 0.951 for AD classification and 0.905 for stroke classification. These findings highlight the importance of calibrated, uncertainty-aware VLMs for trustworthy neuroimaging AI.

neuroscience↗

bioRxiv2025

MAVeRiC-AD: Mixture-of-experts Agentic Vision-Language Ensemble for Robust MRI Classification of Alzheimer's Disease

Robust classification of Alzheimers disease (AD) from structural T1-weighted MRI (T1w) images remains an unmet clinical need, especially when data is acquired at multiple sites that differ in scanning protocols and population demographics. In this paper, we present MAVeRiC-AD (Mixture-of-experts Agent-guided Vision-Language Ensemble for Robust Imaging-based classification of Alzheimers Disease), an agentic framework that dynamically utilizes the optimal inferencing tool for radiological queries to provide relevant answers to the user. In our framework, we tested three specialized models encoded as callable tools: (1) CNN-AD, a 3D DenseNet trained on T1w intensities only; (2) MOE-VLM, a vision-language model that jointly models the T1w with subject-specific demographics (age, sex, site) via a mixture-of-experts (MoE) projection head; (3) Retrieval engine, a similarity-search module that contextualizes a patient against others from the site and reports % prevalence of AD. A light-weight agent analyzes the user request and then routes the input (image, or image + text) to the appropriate tool, aggregates responses and returns the tool response augmented with its confidence derived from conformal prediction. Experiments were conducted using T1w images from the ADNI (N=4,098) and OASIS-3 (N=600) datasets. Single-site training baselines achieved ROC-AUC = 0.79 (CNN) and 0.82 (VLM) on ADNI. When trained jointly on both sites, MOE-VLM surpassed both image-only and standard vision-language models with ROC-AUC = 0.90 on ADNI and 0.81 on OASIS. MAVeRiC-AD demonstrates that agentic orchestration of complementary expert deep models, coupled with explicit demographic conditioning for multi-site data can improve robustness and interpretability of AD image analysis pipelines and serves as a blueprint for scalable, trustworthy clinical AI assistants.

neuroscience↗

bioRxiv2025

Parameter Efficient Fine-tuning of Transformer-based Masked Autoencoder Enhances Resource Constrained Neuroimage Analysis

Recent innovations in artificial intelligence (AI) have increasingly focused on large-scale foundational models that are more general purpose in contrast to conventional models trained to perform specialized tasks. Transformer-based architectures have become the standard backbone in foundation models across data modalities (image, text, audio, video). There has been a keen interest in applying parameter-efficient fine-tuning (PEFT) methods to adapt these models to specialized downstream tasks in language and vision. These methods are particularly essential for medical image analysis where the limited availability of training data could lead to overfitting. In this work, we evaluated different types of PEFT methods on pre-trained vision transformers relative to typical training approaches, such as full fine-tuning and training from scratch. We used a transformer-based masked autoencoder (MAE) framework, to pretrain a vision encoder on T1- weighted (T1-w) brain MRIs. The pretrained vision transformers were then fine-tuned using different PEFT methods that reduced the trainable model parameters to as few as 0.04% of the original model size. Our study shows that: 1. PEFT methods were competitive with or outperformed the reference full fine-tuning approach and outperformed training from scratch, with only a fraction of the trainable parameters; 2. PEFT methods with a 32% reduction in model size boosted Alzheimers disease (AD) classification by 3% relative to full fine-tuning and 11% relative to a 3D CNN, with only 258 training scans; and 3. PEFT methods performed well on diverse neuroimaging tasks including AD and Parkinsons disease (PD) classification, and "brain-age" prediction based on T1-w MRI datasets - a standard benchmark for deep learning models in neuroimaging; 4. smaller model sizes were competitive with larger models in test performance. Our results show the value of adapting foundation models to neuroimaging tasks efficiently and effectively in contrast to training stand- alone special purpose models.

neuroscience↗

bioRxiv2025

Leveraging a Vision-Language Model with Natural Text Supervision for MRI Retrieval, Captioning, Classification, and Visual Question Answering

Large multimodal models are now extensively used worldwide, with the most powerful ones trained on massive, general-purpose datasets. Despite their rapid deployment, concerns persist regarding the quality and domain relevance of the training data, especially in radiology, medical research, and neuroscience. Additionally, healthcare data privacy is paramount when querying models trained on medical data, as is transparency regarding service hosting and data storage. So far, most deep learning algorithms in radiologic research are designed to perform a specific task (e.g., diagnostic classification) and cannot be prompted to perform multiple tasks using natural language. In this work, we introduce a framework based on vector retrieval and contrastive learning to efficiently learn visual brain MRI concepts via natural language supervision. We show how the method learns to identify factors that affect the brain in Alzheimers disease (AD) via joint embedding and natural language supervision. First, we pre-train separate text and image encoders using self-supervised learning, and jointly fine-tune these encoders to develop a shared embedding space. We train our model to perform multiple tasks, including MRI retrieval, MRI captioning, and MRI classification. We show its versatility by developing a retrieval and re-ranking mechanism along with a transformer decoder for visual question answering. Clinical RelevanceBy learning a cross-modal embedding of radiologic features and text, our approach can learn to perform diagnostic and prognostic assessments in AD research as well as to assist practicing clinicians. Integrating medical imaging with clinical descriptions and text prompts, we aim to provide a general, versatile tool for detecting radiologic features described by text, offering a new approach to radiologic research.

neuroscience↗

bioRxiv2024

Counterfactual MRI Generation with Denoising Diffusion Models for Interpretable Alzheimer's Disease Effect Detection

Generative AI models have recently achieved mainstream attention with the advent of powerful approaches such as stable diffusion, DALL-E and MidJourney. The underlying breakthrough generative mechanism of denoising diffusion modeling can generate high quality synthetic images and can learn the underlying distribution of complex, high-dimensional data. Recent research has begun to extend these models to medical and specifically neuroimaging data. Typical neuroimaging tasks such as diagnostic classification and predictive modeling often rely on deep learning approaches based on convolutional neural networks (CNNs) and vision transformers (ViTs), with additional steps to help in interpreting the results. In our paper, we train conditional latent diffusion models (LDM) and denoising diffusion probabilistic models (DDPM) to provide insight into Alzheimers disease (AD) effects on the brains anatomy at the individual level. We first created diffusion models that could generate synthetic MRIs, by training them on real 3D T1-weighted MRI scans, and conditioning the generative process on the clinical diagnosis as a context variable. We conducted experiments to overcome limitations in training dataset size, compute time and memory resources, testing different model sizes, effects of pretraining, training duration, and latent diffusion models. We tested the sampling quality of the disease-conditioned diffusion using metrics to assess realism and diversity of the generated synthetic MRIs. We also evaluated the ability of diffusion models to conditionally sample MRI brains using a 3D CNN-based disease classifier relative to real MRIs. In our experiments, the diffusion models generated synthetic data that helped to train an AD classifier (using only 500 real training scans) -and boosted its performance by over 3% when tested on real MRI scans. Further, we used implicit classifier-free guidance to alter the conditioning of an encoded individual scan to its counterfactual (representing a healthy subject of the same age and sex) while preserving subject-specific image details. From this counterfactual image (where the same person appears healthy), a personalized disease map was generated to identify possible disease effects on the brain. Our approach efficiently generates realistic and diverse synthetic data, and may create interpretable AI-based maps for neuroscience research and clinical diagnostic applications.

bioinformatics↗

bioRxiv2022

Evaluation of Transfer Learning Methods for Detecting Alzheimer's Disease with Brain MRI

Deep neural networks show great promise for classifying brain diseases and making prognostic assessments based on neuroimaging data, but large, labeled training datasets are often required to achieve high predictive accuracy. Here we evaluated a range of transfer learning or pre-training strategies to create useful MRI representations for downstream tasks that lack large amounts of training data, such as Alzheimers disease (AD) classification. To test our proposed pre-training strategies, we analyzed 4,098 3D T1-weighted brain MRI scans from the Alzheimers Disease Neuroimaging Initiative (ADNI) cohort and independently validated with an out-of-distribution test set of 600 scans from the Open Access Series of Imaging Studies (OASIS3) cohort for detecting AD. First, we trained 3D and 2D convolutional neural network (CNN) architectures. We tested combinations of multiple pre-training strategies based on (1) supervised, (2) contrastive learning, and (3) self-supervised learning - using pre-training data within versus outside the MRI domain. In our experiments, the 3D CNN pre-trained with contrastive learning provided the best overall results - when fine-tuned on T1-weighted scans for AD classification - outperformed the baseline by 2.8% when trained with all of the training data from ADNI. We also show test performance as a function of the training dataset size and the chosen pre-training method. Transfer learning offered significant benefits in low data regimes, with a performance boost of 7.7%. When the pre-trained model was used for AD classification, we were able to visualize an improved clustering of test subjects diagnostic groups, as illustrated via a uniform manifold approximation (UMAP) projection of the high-dimensional model embedding space. Further, saliency maps indicate the additional activation regions in the brain scan using pre-training, that then maximally contributed towards the final prediction score.

neuroscience↗

bioRxiv2021

3D Convolutional Neural Networks for Classification of Alzheimer's and Parkinson's Disease with T1-Weighted Brain MRI

Parkinsons disease (PD) and Alzheimers disease (AD) are progressive neurodegenerative disorders that affect millions of people worldwide. In this work, we propose a deep learning approach to classify these diseases based on 3D T1-weighted brain MRI. We analyzed several datasets including the Parkinsons Progression Markers Initiative (PPMI), an independent dataset from the University of Pennsylvania School of Medicine (UPenn), the Alzheimers Disease Neuroimaging Initiative (ADNI), and the Open Access Series of Imaging Studies (OASIS) dataset. The UPenn and OASIS datasets were used as independent test sets to evaluate the model performance during inference. We also implemented a random forest classifier as a baseline model by extracting key radiomics features from the same T1-weighted MRI scans. The proposed 3D convolutional neural network (CNN) model was trained from scratch for the classification tasks. For AD classification, the 3D CNN model achieved an ROC-AUC of 0.878 on the ADNI test set and an average ROC-AUC of 0.789 on the OASIS dataset. For PD classification, the proposed 3D CNN model achieved an ROC-AUC of 0.667 on the PPMI test set and an average ROC-AUC of 0.743 on the UPenn dataset. Model performance was largely maintained when using only 25% of the training dataset. The 3D CNN outperformed the random forest classifier for both the PD and AD tasks. The 3D CNN also generalized better on unseen MRI data from different imaging centers. These approaches show promise for screening of PD and AD patients using only T1-weighted brain MRI, which is relatively widely available. This model with additional validation could also be used to help differentiate between challenging cases of AD and PD when they present with similarly subtle motor and non-motor symptoms.

bioinformatics↗