A generative model for dimensionality reduction with millions of features and few samples

Abstract

MotivationIn this paper, we demonstrate that it is feasible to train a deep generative model for dimensionality reduction with millions of features using few samples, which makes this type of generative model a more versatile alternative to standard methods for dimensionality reduction. Specifically, we hypothesize that for a decoder-only model, the number of training samples required is almost independent of the feature dimensionality in most network architectures. ResultsThrough an extensive set of experiments on synthetic non-linear data, we validate this hypothesis. We also train the model on a downsampled version of the 1000 Genomes Project (1KGP) dataset to further assess its behavior under controlled reductions in sample size. Furthermore, we train a deep generative decoder (DGD) on a curated dataset from the International Cancer Genome Consortium (ICGC), which contains 4.4 million features. It is trained on approximately 4,000 samples and tested on 1,000 samples. The resulting latent representation exhibits clear clustering, and when methods are reduced to the same number of dimensions, it outperforms PCA and VAE for tumor type classification. Additionally, the DGD is computationally efficient and can be trained on a 16GB GPU. Availability and implementationCode is available at https://github.com/cpancott/ReceptiveDGD. Contactcorrado.pancotti@helmholtz-munich.de; akrogh@di.ku.dk Supplementary informationSupplementary data are available with this preprint.