Superior batch alignment and hyper-dimensional cytometry representations allow ultra-sensitive classification of disease phenotypes
Analysis of cytometry data predominantly relies on clustering and dimensionality reduction approaches for computational tractability. This is particularly relevant for modern spectral flow cytometers, which can simultaneously measure an increasingly large number of antibody marker channels. While dimensionality reduction provides for more efficient data processing, this comes at the expense of data loss that may miss subtle patterns among rare cell types that may be critical for disease detection. Maintaining analysis at full dimensions presents opportunities to preserve resolution and provides complete downstream data interpretability. However, a significant obstacle to analysis of cytometry data without dimensionality reduction is the significant batch effects observed in this data between experiment days, operators and equipment types. Here we show a new strategy to denoise batch variation from both flow- and mass-cytometry datasets using an autoencoder neural network architecture. We generated a benchmark flow cytometry dataset in mice to compare this approach to current toolsets and find our approach shows superior preservation of biological signals, whilst also performing batch correction equal to current best methodology. Our batch alignment approach works to such an extent that it becomes practically possible to project batch aligned data into multidimensional space to generate a novel representation of cellular phenotype for downstream model building. This hyperdimensional approach maintains original data resolution without requiring dimensionality reduction, and thus any resultant cell populations that differentiate phenotypes remain fully interpretable. We show with two large clinical datasets that our batch-alignment approach coupled with the multi-dimensional representation successfully detects meaningful patterns in cases where the original analysis methods struggled. This new framework removes some of the inherent technical limitations encountered in the integration of large, multi-batch cytometry datasets and provides a framework for machine learning model building from this modality. An implementation of this framework and an associated web application accompanies this manuscript at http://voxelcoder.cloud. Significance StatementThis work addresses a critical bottleneck in cytometry analysis by introducing a neural network-based approach that effectively removes technical batch variation while preserving biological signals. The novel hyperdimensional representation maintains full data resolution without dimensionality reduction, enabling more sensitive detection of disease-associated cellular signatures than current methods. Importantly, this framework enables reliable batch normalization of fresh samples processed at different times and locations, overcoming the practical constraints of clinical sample collection where simultaneous processing is often impossible. The enhanced sensitivity for identifying pathogenic cellular patterns has immediate implications for biomarker discovery and personalized medicine applications.