Biology subjects

Hallee, L.

Publications and source records attributed to Hallee, L..

3 recordsLinked to original sources

bioRxiv2023

cdsBERT - Extending Protein Language Models with Codon Awareness

AO_SCPLOWBSTRACTC_SCPLOWRecent advancements in Protein Language Models (pLMs) have enabled high-throughput analysis of proteins through primary sequence alone. At the same time, newfound evidence illustrates that codon usage bias is remarkably predictive and can even change the final structure of a protein. Here, we explore these findings by extending the traditional vocabulary of pLMs from amino acids to codons to encapsulate more information inside CoDing Sequences (CDS). We build upon traditional transfer learning techniques with a novel pipeline of token embedding matrix seeding, masked language modeling, and student-teacher knowledge distillation, called MELD. This transformed the pretrained ProtBERT into cdsBERT; a pLM with a codon vocabulary trained on a massive corpus of CDS. Interestingly, cdsBERT variants produced a highly biochemically relevant latent space, outperforming their amino acid-based counterparts on enzyme commission number prediction. Further analysis revealed that synonymous codon token embeddings moved distinctly in the embedding space, showcasing unique additions of information across broad phylogeny inside these traditionally "silent" mutations. This embedding movement correlated significantly with average usage bias across phylogeny. Future fine-tuned organism-specific codon pLMs may potentially have a more significant increase in codon usage fidelity. This work enables an exciting potential in using the codon vocabulary to improve current state-of-the-art structure and function prediction that necessitates the creation of a codon pLM foundation model alongside the addition of high-quality CDS to large-scale protein databases.

bioinformatics↗

bioRxiv2023

Protein-Protein Interaction Prediction is Achievable with Large Language Models

Predicting protein-protein interactions (PPIs) is vital for elucidating fundamental biology, designing peptide therapeutics, and for high-throughput protein annotation. This is particularly relevant in the current biotechnology landscape characterized by the proliferation of protein generative models, which necessitate a high-throughput and generalized PPI predictor for proteins regardless of conventional motifs or known biological functions. Our work addresses this need and provides strong evidence of the utility and reliability of protein language models (pLMs) in learning the PPI objective. We demonstrated that with the use of a sizable balanced dataset, pLMs achieve state-of-the-art performance metrics in PPI prediction on diverse proteins. To generate a dataset that allows for the approximation of these conditions, we implemented a novel synthetic data generation scheme to augment BIOGRID and Negatome datasets. The enhancement of these datasets was then used to fine-tune ProtBERT for PPI prediction to develop a model that we call SYNTERACT (SYNThetic data-driven protein-protein intERACtion Transformer). Our results are compelling, demonstrating 92% accuracy on validated positive and negative interacting pairs derived from 50 different organisms, all of which were excluded from the training phase. In addition to the high metrics, secondary analysis revealed that our synthetic negative data was able to successfully mimic actual negative samples, further reinforcing the integrity of synthetic data additions to PPI datasets. Another notable discovery was the ease in which previously existing PPI datasets could be predicted with simplistic features, calling into question if they can actually inform PPI prediction. We find that the subcellular compartment bias inherent to the compilation of these datasets is learnable with deep learning methods and demonstrate that our approach is not burdened by this disadvantage.

bioinformatics↗

bioRxiv2022

Exploring Phylogenetic Classification and Further Applications of Codon Usage Frequencies

Codons are the repeats of three nucleic acids in genetic material read during translation. 64 total codons exist at different frequencies known to vary between organisms. Codon usage frequencies (CUFs) have recently been used for phylogenetic classification at different discrimination levels. However, the accuracy of phylogenetic classification and applications of this predictive power are not fully elucidated in the current literature. The goal of this project was two-fold: 1.) To increase the accuracy and depth of phylogenetic classification models using CUFs in literature. 2.) To explore the potential application of identifying open reading frames (ORFs) with CUFs. To increase the accuracy of classification models GridSearchCV, TensorFlow, and keras were used to design an improved Artificial Neural Network than the relevant example in the literature. Commonly used predictors were explored in an ensemble format that performs even better than the improved neural network. To explore a more discriminatory and phylogenetically deep classification K Means was used to look at possible clustering structures in the CUF data. To identify ORFs the codon frequencies of each potential ORF are compared to the frequencies of an organism of choice with a multinomial goodness-of-fit test. With correct optimization, these tests can reject possible ORFs with high confidence. In addition to these goals, the codons were ranked in terms of importance for classification with lasso regression and random forests feature ranking. This not only highlights exciting biology related to tRNA concentrations and the variance thereof, but is also helpful for optimizing the statistical tests for ORF identification.

bioinformatics↗