Biology subjects

Betke, M.

Publications and source records attributed to Betke, M..

2 recordsLinked to original sources

bioRxiv2024

FourierMIL: Fourier filtering-based multiple instance learning for whole slide image analysis

Recent advancements in computer vision, driven by convolutional neural network, multilayer perceptron and transformer architectures, have significantly improved the analysis on natural images. Despite their potential, the application of these architectures in digital pathology, specifically for analyzing gigapixel-resolution whole-slide images (WSIs), remains challenging due to the extensive and variable sizes of these images. Here we present a multiple instance learning framework that leverages the discrete Fourier transform and learns from WSIs. Dubbed as FourierMIL, our framework is designed to capture both global and local dependencies within WSIs. To validate the efficacy of our model, we conducted extensive experiments on a prevalent computational pathology challenge: tumor classification. Our results demonstrate that FourierMIL outperforms existing state-of-the-art methods, marking a significant advancement in the field of digital pathology and highlighting the potential of attention-free architectures in managing the complexities related to WSI analysis. The code will be released for public access upon the manuscripts acceptance.

bioinformatics↗

bioRxiv2023

Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival

Multimodal machine learning models are being developed to analyze pathology images and other modalities, such as gene expression, to gain clinical and biological in-sights. However, most frameworks for multimodal data fusion do not fully account for the interactions between different modalities. Here, we present an attention-based fusion architecture that integrates a graph representation of pathology images with gene expression data and concomitantly learns from the fused information to predict patient-specific survival. In our approach, pathology images are represented as undirected graphs, and their embeddings are combined with embeddings of gene expression signatures using an attention mechanism to stratify tumors by patient survival. We show that our framework improves the survival prediction of human non-small cell lung cancers, out-performing existing state-of-the-art approaches that lever-age multimodal data. Our framework can facilitate spatial molecular profiling to identify tumor heterogeneity using pathology images and gene expression data, complementing results obtained from more expensive spatial transcriptomic and proteomic technologies.

bioinformatics↗