Decisive Roles of Sequence Distributions in the Generalizability of de novo Deep Learning Models for RNA Secondary Structure Prediction

Abstract

Taking sequences as the only inputs, the class of de novo deep learning (DL) models for RNA secondary structure prediction has achieved far superior performances than traditional algorithms. However, key questions remain over the statistical underpinning of such models that make no use of physical laws or co-evolutionary information. We present a quantitative study of the capacity and generalizability of a series of de novo DL models, with a minimal two-module architecture and no post-processing, under varied distributions of the seen and unseen sequences. Our DL models outperform existing methods on commonly used benchmark datasets and demonstrate excellent learning capacities under all sequence distributions. These DL models generalize well over non-identical unseen sequences, but the generalizability degrades rapidly as the sequence distributions of the seen and unseen datasets become dissimilar. Examinations of RNA family-specific behaviors manifest not only disparate familydependent performances but substantial generalization gaps within the same family. We further determine how model generalization decreases with the decrease of sequence similarity via pairwise sequence alignment, providing quantitative insights into the limitations of statistical learning. Model generalizability thus poses a major hurdle for practical uses of de novo DL models and several tenable avenues for future advances are discussed.