Integrating cellular graph embeddings with tumor morphological features to predict in-silico spatial transcriptomics from H&E images

Abstract

Spatial transcriptomics allows precise RNA abundance measurement at high spatial resolution, linking cellular morphology with gene expression. We present a novel deep learning algorithm predicting local gene expression from histopathology images. Our approach employs a graph isomorphism neural network capturing cell-to-cell interactions in the tumor microenvironment and a Vision Transformer (CTransPath) for obtaining the tumor morphological features. Using a dataset of 30,612 spatially resolved gene expression profiles matched with histopathology images from 23 breast cancer patients, we identify 250 genes, including established breast cancer biomarkers, at a 100 {micro}m resolution. Additionally, we co-train our algorithm on spatial spot-level transcriptomics from 10x Visium breast cancer data along with another variant of our algorithm on TCGA-BRCA bulk RNA Seq. data, yielding mutual benefits and enhancing predictive accuracy on both these datasets. This work enables image-based screening for molecular biomarkers with spatial variation, promising breakthroughs in cancer research and diagnostics.