Neighborhood nonnegative matrix factorization identifies patterns and spatially-variable genes in large-scale spatial transcriptomics data

Abstract

Tissues consist of multi-cellular neighborhoods in which different cell types express correlated gene programs due to shared signaling environments. Methods for identifying these spatial neighborhoods may be powerful, but currently do not scale to existing data sets of millions of cells and often artificially divide tissues into distinct neighborhoods with hard borders. To better identify multi-cellular microenvironments with shared gene programs in large-scale spatial genomics data, we developed a method that combines nonnegative matrix factorization (NMF) with Gaussian smoothing across cells in space. Our spatially-aware dimension reduction, neighborhood NMF (NNMF), identifies known and unknown interactions among diverse cell types organized into complex patterns, from localized structures to broad tissue regions. NNMF has many advantages over currently available methods, including the ability to run on modern large-scale data with thousands of features and multiple tissue samples and with millions of cells. Furthermore, our method is based on probabilistic NMF, which produces soft clusters of landscape signatures that can be understood as overlapping spatially-organized multicellular gene expression programs, allowing more biologically-complete interpretations than overly-simplistic hard clustering methods. In a benchmark dataset of a diverse set of spatial gene expression data with expert tissue labels, compared against related methods, NNMF with K-nearest neighbors clustering shows excellent performance even on hard clustering tasks. On MERFISH human colorectal cancer data, NNMF identifies several immunologically relevant multicellular interaction networks and scales to these data sets with million of cells. NNMF is implemented as an R package available at https://github.com/ragnhildlaursen/NNMF.