CircSSNN: circRNA-binding site prediction via sequence self-attention neural networks with pre-normalization

Abstract

Circular RNAs (circRNAs) play a significant role in some diseases by acting as transcription templates. Therefore, analyzing the interaction mechanism between circRNA and RNA-binding proteins (RBPs) has far-reaching implications for the prevention and treatment of diseases. Existing models for circRNA-RBP identification most adopt CNN, RNN, or their variants as feature extractors. Most of them have drawbacks such as poor parallelism, insufficient stability, and inability to capture long-term dependence. To address these issues, we designed a Seq_transformer module to extract deep semantic features and then propose a CircRNA-RBP identification model based on Sequence Self-attention with Pre-normalization. We test it on 37 circRNA datasets and 31 linear RNA datasets using the same set of hyperparameters, and the overall performance of the proposed model is highly competitive and, in some cases, significantly out-performs state-of-the-art methods. The experimental results indicate that the proposed model is scalable, transformable, and can be applied to a wide range of applications without the need for task-oriented fine-tuning of parameters. The code is available at https://github.com/cc646201081/CircSSNN. Author summaryIn this paper, we propose a new method completely using the self-attention mechanism to capture deep semantic features of RNA sequences. On this basis, we construct a CircSSNN model for the cirRNA-RBP identification. The proposed model constructs a feature scheme by fusing circRNA sequence representations with statistical distributions, static local context, and dynamic global context. With a stable and efficient network architecture, the distance between any two positions in a sequence is reduced to a constant, so CircSSNN can quickly capture the long-term dependence and extract the deep semantic features. Experiments on 37 circRNA datasets show that the proposed model has overall advantages in stability, parallelism, and prediction performance. Keeping the network structure and hyperparameters unchanged, we directly apply CircSSNN to linRNA datasets. The favorable results show that CircSSNN can be transformed simply and efficiently without task-oriented tuning. In conclusion, CircSSNN can serve as an appealing circRNA-RBP identification tool with good identification performance, excellent scalability, and wide application scope, which is expected to reduce the professional threshold required for hyperparameter tuning in bioinformatics analysis.