evoCancerGPT: Generating Zero-Shot Single-Cell and Single-Sample Cancer Progression Through Transfer Learning

Abstract

Cancer evolution is driven by complex changes in gene expression as cells transition and change states during tumorigenesis. Single-cell RNA sequencing has provided snapshot insights into how the transcriptomics of tumors evolve, but whether the existing knowledge can be used to reliably learn and generate the patterns behind the evolution of cancers remains unknown. Here, we introduce evoCancerGPT, a generative pre-trained transformer decoder-only single-cell foundation model designed to forecast future gene expression profiles in cancer evolution by leveraging previous cell states at the level of single patients. This model integrates the continuous gene expression data of each cell to create a comprehensive representation of a cell token. Training sentences are constructed for each cancer type, each patient and each cell type separately, ordered via inferred pseudotime algorithms, using 2.76 million cell tokens, each with 12,639 genes, spanning 7 cancer types. By learning from long-range dependencies between cells arranged in pseudotime from a large corpus of data, evoCancerGPT captures key transitions in cancer evolution, achieving high concordance to ground truth trajectories and outperforming linear and scGPT baselines in held-out test samples in low-context scenarios. Our work suggests evoCancerGPTs potential utility in characterizing tumor progression at a single-cell and single-patient level and ultimately contributing to more personalized cancer care.