Context-Aware Generative Models for Multi-Domain Proteins using Transformers

Abstract

MotivationBeing able to artificially design novel proteins of desired function is pivotal in many biological and biomedical applications. Generative statistical modeling has recently emerged as a new paradigm for designing amino acid sequences, including in particular models and embedding methods borrowed from Natural Language Processing (NLP). However, most approaches target single proteins or protein domains, and do not take into account any functional specificity or interaction with the context. To extend beyond current computational strategies, we develop a method for generating protein domain sequences intended to interact with another protein domain. Using data from natural multi-domain proteins, we cast the problem as a translation problem from a given interactor domain to the new domain to be generated, i.e. we generate artificial partner sequences conditional on an input sequence. ResultsEvaluating our models quality using diverse metrics, in part related to distinct biological questions, we show that our method outperforms state-of-the-art shallow auto-regressive strategies. We also explore the possibility of fine-tuning pre-trained large language models for the same task and of using Alphafold 2 for assessing the quality of sampled sequences.