DNAPerceiver and ProteinPerceiver: deep learning models for gene and protein expression prediction from DNA and amino-acid sequences
Background and ObjectiveThe functions of an organism and its biological processes result from the expression of genes and proteins. Therefore quantifying and predicting mRNA and protein levels is a crucial aspect of scientific research. Concerning the prediction of mRNA levels, the available approaches use the sequence straddling the Transcription Start Site (TSS) as input to neural networks. The State-of-the-art models (e.g., Xpresso and Basenjii) predict mRNA levels exploiting Convolutional (CNN) or Long Short Term Memory (LSTM) Networks. However, CNN prediction depends on convolutional kernel size, and LSTM suffers from capturing long-range dependencies in the sequence. Concerning the prediction of protein levels, as far as we know, there is no model for predicting protein levels by exploiting the gene or protein sequences. MethodsHere, we exploit a new model type (called Perceiver) for mRNA and protein level prediction, exploiting a Transformer-based architecture with an attention module to attend to long-range interactions in the sequences. In addition, the Perceiver model overcomes the quadratic complexity of the standard Transformer architectures. This works contributions are 1. DNAPerceiver model to predict mRNA levels from the sequence straddling the TSS; 2. ProteinPerceiver model to predict protein levels from the protein sequence; 3. Protein&DNAPerceiver model to predict protein levels from TSS-straddling and protein sequences. ResultsThe models are evaluated on cell lines, mice, glioblastoma, and lung cancer tissues. The results show the effectiveness of the Perceiver-type models in predicting mRNA and protein levels. ConclusionsThis paper presents a Perceiver architecture for mRNA and protein level prediction. In the future, inserting regulatory and epigenetic information into the model could improve mRNA and protein level predictions. The source code is freely available at https://github.com/MatteoStefanini/DNAPerceiver Graphical Abstract O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=102 SRC="FIGDIR/small/508821v2_ufig1.gif" ALT="Figure 1"> View larger version (23K): org.highwire.dtl.DTLVardef@1920a69org.highwire.dtl.DTLVardef@e97761org.highwire.dtl.DTLVardef@19c83cborg.highwire.dtl.DTLVardef@90c3d3_HPS_FORMAT_FIGEXP M_FIG C_FIG HighlightsO_LIPredicting mRNA and protein levels from DNA and protein sequences is crucial in clinical applications. C_LIO_LIA transformer-based architecture with asymmetric attention (Perceiver) is exploited for mRNA and protein level prediction. C_LIO_LIThe Perceiver architecture attends to longer range interactions compared to Transformer, CNN, and LSTM. C_LIO_LIThe proposed model achieves state-of-the-art performance for mRNA level prediction. C_LIO_LITo the best of our knowledge, the protein level prediction task is addressed. C_LIO_LIThe proposed model is tested on glioblastoma and lung cancer tissues. C_LI