RNAtomy: A Structure-Aware RNA Foundation Model for Transferable Structural and Functional Representations
RNA plays a critical and diverse role in gene expression, catalysis, regulation, and molecular recognition. This functional versatility arises from the interplay between nucleotide sequence, structural organization, and cellular context. Nevertheless, most RNA foundation models are pretrained primarily using sequence or secondary-structure information, while the systematic integration of three-dimensional geometric and conformational features remains comparatively underexplored. Here, we introduce RNAtomy, a self-supervised RNA foundation model designed to learn the structural anatomy of RNA from inter-nucleotide geometry and residue-level conformational descriptors. Its design has three main components: 1) joint masked reconstruction of pairwise geometry and residue-level descriptors to connect spatial organization with local conformation; 2) a hybrid Mamba-Transformer backbone in which Mamba blocks efficiently propagate global context, while shifted-window attention compensates for the absence of explicit content-dependent token interactions in Mamba by modeling relationships within local regions of the pairwise map; and 3) axis-aware feature-wise linear modulation that maps nucleotide-level conformations to the row and column axes of pairwise geometry. We assess RNAtomy across seven diverse tasks involving structural and functional predictions at the nucleotide, nucleotide-pair, and RNA levels. RNAtomy achieved particularly strong performance on structural tasks, outperforming established RNA foundation models and task-specific approaches in contact and long-range base-pair prediction. Its representations also transferred effectively to interaction-related tasks, despite the substantial differences between these applications and the structural pretraining objective. On functional tasks, RNAtomy achieved superior performance in translation-related prediction and regulatory-element identification. Notably, RNAtomy achieved these results despite being pretrained on a substantially smaller training set than the RNA foundation models used as baselines. Together, these results highlight how rich structural information, when learned through an expressive architecture, can support transferable RNA representations across biologically distinct structural and functional applications.