Biology subjects

Hie, B.

Publications and source records attributed to Hie, B..

15 recordsLinked to original sources

bioRxiv2026

Few-shot regulatory sequence design with a genomic language model

A central challenge in generative biology is translating knowledge learned from natural sequences and experiments into designs with desired functions. In-context learning (ICL) is an emergent capability of large language models that allows them to adapt to new tasks from a few examples without additional training, enabling these models to become general-purpose learners. Here, we show that the genome language model, Evo 2, exhibits generative ICL in biological sequence design. Prompting with just tens of examples conditions Evo 2 to generate novel and diverse regulatory DNA sequences without any additional training. Mechanistic analysis suggests that genomic ICL may originate from induction, similar to phenomena observed in natural language. Reinforcement learning leveraging sequence-to-function models as experiment-derived rewards further strengthens ICL. We first validate this approach by generating enhancers that drive cell-type-specific gene expression in three human cell lines. Extending this approach in vivo, enhancers designed using human regulatory element data selectively label layer 5 extratelencephalic neurons in the mouse cortex with high specificity and stronger activity than existing enhancers. Our study provides evidence of emergent properties in genomic sequence models and introduces a promising approach for improving functional conditioning in generative biology.

bioengineering↗

bioRxiv2026

Efficient biological design via genomic in-context learning

In-context learning (ICL), the ability to perform new tasks from a few examples supplied in a prompt, is a defining emergent capability of large language models. Here, we show that ICL emerges in the genome language model Evo 2 and enables biological design. With as few as four examples in the prompt and no additional training, Evo 2 generates realistic metal-binding proteins, de novo antibodies, protein-RNA systems, and entire bacteriophage genomes. For {Phi}X174-like bacteriophage design, ICL surpasses previously fine-tuned models on computational measures of design quality and diversity. Beyond {Phi}X174, conditioning the model with different phage prompts enables us to efficiently design MS2 and M13 genomes in silico without fine-tuning. Iteratively updating the prompt with newly generated phage genomes, which we refer to as genomic chain-of-thought, further improves the diversity of designs while maintaining quality metrics. Paired with an inference engine that accelerates generation up to 35-fold and extends practicable context to a megabase, these results establish ICL as a versatile and efficient paradigm for biological sequence design.

bioinformatics↗

bioRxiv2026

Coevolutionary mining of prokaryotic non-coding elements with a genome language model

Microbial genomes encode compact molecular machines and diverse non-coding RNAs (ncRNAs) essential to gene regulation, pathogenesis, and many foundational biotechnologies. However, annotation remains largely protein-centric and homology-driven. Here, we introduce Minerva, a framework for coevolutionary mining that uses genome language models to predict local interactions directly from sequence as two-dimensional maps. Introducing two complementary techniques, categorical Jacobian fingerprinting and interaction heads, we demonstrate fast, accurate, alignment-free prediction of ncRNA base-pairing, monomeric protein contacts, and repetitive sequence motifs. Applied to 150 bacterial genomes, Minerva recovers known systems and predicts that 84.3% of predicted intergenic base-pairing falls outside of known annotations. In Pseudomonas, we find that the widespread TwoAYGGAY ncRNA family carries large secondary-structure extensions and is often flanked by short upstream repetitive motifs and larger downstream genomic repeats. Interpreting the coevolution maps, we find that Minerva emergently detects open-reading-frame (ORF) signatures at the DNA level despite never being trained to do so. In prophages within these genomes, we discover that Unknown Group 27 (UG27) reverse transcriptase systems encode arrays of structurally conserved yet sequence-diverse ncRNAs that template complementary DNA (cDNA) hairpin products. Together, these results establish coevolutionary mining as a scalable route to genome annotation and biological discovery across the rapidly expanding microbial universe.

bioinformatics↗

bioRxiv2026

A high-level programming language for generative biology with Proto

Programmable composition of complex systems is a longstanding goal of biological research. Generative modeling has improved the reliability of computational design, but existing methods are highly specialized and are difficult to extend or compose. Here, we introduce Proto, a high-level programming language for generative biology. By composing a small set of abstract primitives into structured programs, Proto encodes generative design campaigns across diverse modalities and scales--spanning DNA, RNA, proteins, ligands, and their interactions. Proto readily incorporates predictive models into generative workflows, which we leveraged to design alternatively spliced introns with experimental validation in human cell lines. Proto is natively multi-objective, enabling the design of promoter-repressor pairs with leading experimental success rates for synthetic protein-DNA design. Alongside AI agents, Proto enables the specification of complex pathways and regulatory logic through natural language instructions. We openly release Proto, including software infrastructure and user interfaces, to enable widespread access to generative biological programming.

Synthetic Biology↗

bioRxiv2024

Single-Sequence, Structure Free Allosteric Residue Prediction with Protein Language Models

Large language models trained on protein amino acid sequences have shown the ability to learn general coevolutionary relationships at scale, which in turn contain useful structural and functional information. Here we show that attention maps, matrices of learned pairwise relationships between residues, also include information about allostery. This enables prediction of allosteric relationships with no task-specific training, requiring only a single input sequence and no structural information. Attention maps outperform state-of-the-art structure-based and sequence coevolution-based allosteric residue prediction models on a well-curated benchmark set of 24 allosteric proteins. For K-Ras, an allosterically regulated GTPase, attention maps correlate best with allosteric residues influencing binding identified in deep mutational scanning data. For the beta-2 adrenergic receptor, an allosterically regulated GPCR, attention maps correlate best when compared to experimental alanine-scanning mutational data identifying allosteric relationships influencing signaling. These results enable allosteric relationship prediction in a single-sequence, structure-free manner.

biochemistry↗

bioRxiv2024

Aligning protein generative models with experimental fitness via Direct Preference Optimization

Generative models trained on unlabeled protein datasets have demonstrated a remarkable ability to predict some biological functions without any task-specific training data. However, this capability does not extend to all relevant functions and, in many cases, the unsupervised model still underperforms task-specific, supervised baselines. We hypothesize that this is due to a fundamental "alignment gap" in which the rules learned during unsupervised training are not guaranteed to be related to the function of interest. Here, we demonstrate how to provide protein generative models with useful task-specific information without losing the rich, general knowledge learned during pretraining. Using an optimization task called Direct Preference Optimization (DPO), we align a structure-conditioned language model to generate stable protein sequences by encouraging the model to prefer stabilizing over destabilizing variants given a protein backbone structure. Our resulting model, ProteinDPO, is the first structure-conditioned language model preference-optimized to experimental data. ProteinDPO achieves competitive stability prediction and consistently outperforms both unsupervised and finetuned versions of the model. Notably, the aligned model also performs well in domains beyond its training data to enable absolute stability prediction of large proteins and binding affinity prediction of multi-chain complexes, while also enabling single-step stabilization of diverse backbones. These results indicate that ProteinDPO has learned generalizable information from its biophysical alignment data.

biophysics↗

bioRxiv2022

A high-level programming language for generative protein design

Combining a basic set of building blocks into more complex forms is a universal design principle. Most protein designs have proceeded from a manual bottom-up approach using parts created by nature, but top-down design of proteins is fundamentally hard due to biological complexity. We demonstrate how the modularity and programmability long sought for protein design can be realized through generative artificial intelligence. Advanced protein language models demonstrate emergent learning of atomic resolution structure and protein design principles. We leverage these developments to enable the programmable design of de novo protein sequences and structures of high complexity. First, we describe a high-level programming language based on modular building blocks that allows a designer to easily compose a set of desired properties. We then develop an energy-based generative model, built on atomic resolution structure prediction with a language model, that realizes all-atom structure designs that have the programmed properties. Designing a diverse set of specifications, including constraints on atomic coordinates, secondary structure, symmetry, and multimerization, demonstrates the generality and controllability of the approach. Enumerating constraints at increasing levels of hierarchical complexity shows that the approach can access a combinatorially large design space.

synthetic biology↗

bioRxiv2022

Language models of protein sequences at the scale of evolution enable accurate structure prediction

Artificial intelligence has the potential to open insight into the structure of proteins at the scale of evolution. It has only recently been possible to extend protein structure prediction to two hundred million cataloged proteins. Characterizing the structures of the exponentially growing billions of protein sequences revealed by large scale gene sequencing experiments would necessitate a break-through in the speed of folding. Here we show that direct inference of structure from primary sequence using a large language model enables an order of magnitude speed-up in high resolution structure prediction. Leveraging the insight that language models learn evolutionary patterns across millions of sequences, we train models up to 15B parameters, the largest language model of proteins to date. As the language models are scaled they learn information that enables prediction of the three-dimensional structure of a protein at the resolution of individual atoms. This results in prediction that is up to 60x faster than state-of-the-art while maintaining resolution and accuracy. Building on this, we present the ESM Metage-nomic Atlas. This is the first large-scale structural characterization of metagenomic proteins, with more than 617 million structures. The atlas reveals more than 225 million high confidence predictions, including millions whose structures are novel in comparison with experimentally determined structures, giving an unprecedented view into the vast breadth and diversity of the structures of some of the least understood proteins on earth.

synthetic biology↗

bioRxiv2022

Learning inverse folding from millions of predicted structures

We consider the problem of predicting a protein sequence from its backbone atom coordinates. Machine learning approaches to this problem to date have been limited by the number of available experimentally determined protein structures. We augment training data by nearly three orders of magnitude by predicting structures for 12M protein sequences using AlphaFold2. Trained with this additional data, a sequence-to-sequence transformer with invariant geometric input processing layers achieves 51% native sequence recovery on structurally held-out backbones with 72% recovery for buried residues, an overall improvement of almost 10 percentage points over existing methods. The model generalizes to a variety of more complex tasks including design of protein complexes, partially masked structures, binding interfaces, and multiple states.

systems biology↗

bioRxiv2020

GM-CSF differentiation of human monocytes stabilizes macrophage state via oxidative signaling

Macrophages are central mediators of immunity that integrate diverse signals derived from differentiation cues, tissue location, and disease. Controlling macrophage state and function is an appealing therapeutic objective across many diseases including cancer, atherosclerosis, and tuberculosis. Despite the growing appreciation for the in vivo complexity of macrophage state, existing in vitro models of human monocyte-derived macrophages have used a limited number of individual perturbations to explore the complex phenotypic space that macrophages can occupy. Here, we leverage a tiered differentiation, activation, and stimulation strategy to generate libraries of in vitro monocyte-derived macrophages and examine the in vitro state space of macrophage function using high-dimensional technologies. Our tiered experimental approach further revealed a striking relationship between GM-CSF differentiation and IL-10 production. Cells that were differentiated with GM-CSF produced very low or undetectable levels of IL-10 independent of activation or stimulation condition. To nominate candidate regulators of this IL-10 response, we leverage unbiased single-cell mRNA sequencing to identify transcriptional networks associated with GM-CSF-derived cells. Using these data, we identify oxidative signaling pathways as upregulated in GM-CSF derived cells and demonstrate that scavenging of oxidative radicals can enhance IL-10 production in these cells. Collectively, these data underscore the complexity of monocyte-derived macrophage state over time and highlight a dominant role for GM-CSF in tuning macrophage inflammatory phenotype, metabolic state, and plasticity.

immunology↗

bioRxiv2020

Learning with uncertainty for biological discovery and design

Machine learning that generates biological hypotheses has transformative potential, but most learning algorithms are susceptible to pathological failure when exploring regimes beyond the training data distribution. A solution is to quantify prediction uncertainty so that algorithms can gracefully handle novel phenomena that confound standard methods. Here, we demonstrate the broad utility of robust uncertainty prediction in biological discovery. By leveraging Gaussian process-based uncertainty prediction on modern pretrained features, we train a model on just 72 compounds to make predictions over a 10,833-compound library, identifying and experimentally validating compounds with nanomolar affinity for diverse kinases and whole-cell growth inhibition of Mycobacterium tuberculosis. We show how uncertainty facilitates a tight iterative loop between computation and experimentation, improves the generative design of novel biochemical structures, and generalizes across disparate biological domains. More broadly, our work demonstrates that uncertainty should play a key role in the increasing adoption of machine learning algorithms into the experimental lifecycle.

bioinformatics↗

bioRxiv2019

Schema: A general framework for integrating heterogeneous single-cell modalities

A complete understanding of biological processes requires synthesizing information across heterogeneous modalities, such as age, disease status, or gene/protein expression. Until recently, single-cell profiling experiments could measure only a single modality, leading to analysis focused on integrating information across separate experiments. However, researchers can now measure multiple modalities simultaneously in a single experiment, providing a new data paradigm that enables biological discovery but also requires new conceptual and analytic models. We therefore present Schema, an algorithm that leverages a principled metric learning strategy to synthesize multimodal information from the same experiment. To demonstrate the flexibility and power of our approach, we use Schema to infer cell types by integrating gene expression and chromatin accessibility data, perform differential gene expression analysis while accounting for batch effects and developmental age, estimate evolutionary pressure on peptide sequences, and synthesize spliced and unspliced mRNA data to infer cell differentiation. Schema can synthesize arbitrarily many modalities and capture sophisticated relationships between them, is computationally efficient, and provides a valuable conceptual model for exploring and understanding complex biology.

bioinformatics↗

bioRxiv2019

Coexpression uncovers a unified single-cell transcriptomic landscape

Single-cell transcriptomic studies of diverse and complex systems are becoming ubiquitous. Algorithms now attempt to integrate patterns across these studies by removing all study-specific information, without distinguishing unwanted technical bias from relevant biological variation. Integration remains difficult when capturing biological variation that is distributed across studies, as when combining disparate temporal snapshots into a panoramic, multi-study trajectory of cellular development. Here, we show that a fundamental analytic shift to gene coexpression within clusters of cells, rather than gene expression within individual cells, balances robustness to bias with preservation of meaningful inter-study differences. We leverage this insight in Trajectorama, an algorithm which we use to unify trajectories of neuronal development and hematopoiesis across studies that each profile separate developmental stages, a highly challenging task for existing methods. Trajectorama also reveals systems-level processes relevant to disease pathogenesis within the microglial response to myelin injury. Trajectorama benefits from efficiency and scalability, processing nearly one million cells in around an hour.

bioinformatics↗

bioRxiv2019

Geometric Sketching Compactly Summarizes the Single-Cell Transcriptomic Landscape

Large-scale single-cell RNA-sequencing (scRNA-seq) studies that profile hundreds of thousands of cells are becoming increasingly common, overwhelming existing analysis pipelines. Here, we describe how to enhance and accelerate single-cell data analysis by summarizing the transcriptomic heterogeneity within a data set using a small subset of cells, which we refer to as a geometric sketch. Our sketches provide more comprehensive visualization of transcriptional diversity, capture rare cell types with high sensitivity, and accurately reveal biological cell types via clustering. Our sketch of umbilical cord blood cells uncovers a rare subpopulation of inflammatory macrophages, which we experimentally validated in vitro. The construction of our sketches is extremely fast, which enabled us to accelerate other crucial resource-intensive tasks such as scRNA-seq data integration. We anticipate that our algorithm will become an increasingly essential step when sharing and analyzing the rapidly-growing volume of scRNA-seq data and help enable the democratization of single-cell omics.

bioinformatics↗

bioRxiv2018

Fine-mapping cis-regulatory variants in diverse human populations

Genome-wide association studies (GWAS) are a powerful approach for connecting genotype to phenotype. Most GWAS hits are located in cis-regulatory regions, but the underlying causal variants and their molecular mechanisms remain unknown. To better understand human cis-regulatory variation, we mapped quantitative trait loci for chromatin accessibility (caQTLs)--a key step in cis-regulation--in 1000 individuals from 10 diverse populations. Most caQTLs were shared across populations, allowing us to leverage the genetic diversity to fine-map candidate causal regulatory variants, several thousand of which have been previously implicated in GWAS. In addition, many caQTLs that affect the expression of distal genes also alter the landscape of long-range chromosomal interactions, suggesting a mechanism for long-range expression QTLs. In sum, our results show that molecular QTL mapping integrated across diverse populations provides a high-resolution view of how worldwide human genetic variation affects chromatin accessibility, gene expression, and phenotype.

molecular biology↗