Jointly leveraging spatial transcriptomics and deep learning models for pathology image annotation improves cell type identification over either approach alone.

Abstract

Spatial transcriptomics technologies have recently emerged as a powerful tool for measuring spatially resolved gene expression directly in tissues sections, revealing cell types and their dysfunction in unprecedented detail. However, spatial transcriptomics technologies are limited in their ability to separate transcriptionally similar cell types and can suffer further difficulties identifying cell types in slide regions where transcript capture is low. Here, we describe a conceptually novel methodology that can computationally integrate spatial transcriptomics data with cell-type-informative paired tissue images, obtained from, for example, the reverse side of the same tissue section, to improve inferences of tissue cell type composition in spatial transcriptomics data. The underlying statistical approach is generalizable to any spatial transcriptomics protocol where informative paired tissue images can be obtained. We demonstrate a use case leveraging cell-type-specific immunofluorescence markers obtained on mouse brain tissue sections and a use case for leveraging the output of AI annotated H&E tissue images, which we used to markedly improve the identification of clinically relevant immune cell infiltration in breast cancer tissue. Thus, combining spatial transcriptomics data with paired tissue images has the potential to improve the identification of cell types and hence to improve the applications of spatial transcriptomics that rely on accurate cell type identification.