Weak supervision of H&E slides reveals systems-level biology and functional states that govern therapeutic resistance

Abstract

Precision oncology lacks scalable methods to identify the mechanisms that mediate therapeutic resistance for individual patients. Resistance often arises from focal cellular niches that are obscured by bulk profiling and costly to resolve with multi-omics. Here, we show that deep learning (DL), applied to routine histology, can localize focal tissue regions enriched for therapeutically relevant biology. Using 3111 breast cancer H&E slides with matched bulk transcriptomics, we trained weakly-supervised DL models to infer activities of immune, metabolic, and tumor-intrinsic phenotypes implicated in therapeutic resistance (AUROC>0.80; PCC>0.64). Accurate inference of these phenotypes should identify tissue regions enriched for the corresponding biological signal. Therefore, we validated phenotype inference and spatial localization with complementary analyses. Tissue-matched multiplexed immunofluorescence showed concordance between inferred immune states and corresponding cell fractions (p=0.006-0.106). Across multi-institutional cohorts, model-derived phenotypes recovered expected relationships with therapeutic outcomes (p<0.045). Finally, in a blinded evaluation, pathologists confirmed that model-derived high-attention regions were enriched for phenotype-specific morphology (p<2.408*10-5). Because evaluated phenotypes represent diverse mechanisms of resistance across therapeutic modalities, these findings provide a foundation for resistance-directed localization using therapeutic outcomes as supervision. By directing deep profiling toward model-prioritized regions, this framework could enable scalable nomination of candidate mediators of resistance for subsequent functional validation across real-world patient populations. One sentence summaryWeakly supervised deep learning localizes focal tissue regions enriched for therapeutically relevant biology in routine histology, thus offering a scalable strategy to study therapeutic resistance across large patient populations.