Biology subjects

Ektefaie, Y.

Publications and source records attributed to Ektefaie, Y..

9 recordsLinked to original sources

bioRxiv2026

Autonomous generalizability auditing of scientific foundation models

Determining where scientific foundation models generalize beyond their development data is essential for reliable use, but aggregate benchmarks obscure performance limits and rigorous audits are difficult to conduct routinely. Here, we introduce AutoSPECTRA, a coding-agent protocol for systematic generalizability auditing. To inform its design, we reviewed 38 studies across five biological domains and evaluated coding agents, which generated hypotheses and empirical analyses but applied inconsistent standards for judging evidence. We then structured AutoSPECTRA around measurable generalization axes, prespecified analyses, controlled comparisons and independent confirmation. Across four biological foundation models, audits linked performance to protein sequence composition and relatedness to pretraining data (ESMFold2), representation of perturbations in training (STATE), effective spatial resolution (CONCH), and residue-level genomic, evolutionary and structural context (AlphaMissense). Agents also evaluated alternative explanations and rejected unsupported hypotheses. AutoSPECTRA provides a systematic approach to discovering and testing the limits of scientific foundation models.

bioinformatics↗

bioRxiv2026

Improving Generalizability in Whole-Cell Antibiotic Discovery Through Active Learning

Machine learning (ML) has accelerated molecular discovery, yet training models to generalize to out-of-distribution (OOD) chemical spaces remains fundamentally constrained by the high cost of experimental validation. In antibiotic discovery, where whole-cell phenotypic high throughput screening (HTS) is resource-intensive, iterative ML-guided compound selection - or Active Learning (AL) - offers a pathway to efficiently navigate available chemical spaces. However, the algorithmic tradeoffs between prioritizing compound novelty (exploration), predicted bioactivity (exploitation), and their impact on OOD generalizability remain unresolved for noisy, whole-cell biological systems. In this work, we systematically evaluate three AL strategies for whole-cell bacterial bioactivity and benchmark their effects on model accuracy, hit rate, and OOD performance. Using retrospective simulations on Mycobacterium tuberculosis HTS data, we identify an optimal AL strategy that balances predicted hit/non-hit novelty with overall hit rate. We then integrate the strategy in a closed-loop Borrelia burgdorferi antibiotic discovery HTS campaign. The AL-guided approach successfully increased the experimental screening hit rate five-fold (from a 0.2% rate within investigator-selected plates to 1.0%). Further, when the trained model was applied in prospective in silico selection of highly diverse compounds across multiple bacterial species, the AL-trained whole-cell inhibition predictor demonstrates 53-fold enrichment over investigator-directed screening (11.0% experimental validation of predicted hits). Of these, 100% demonstrated the intended narrow spectrum activity for Borrelia burgdorferi. These results demonstrate that calibrated AL strategies can overcome data acquisition bottlenecks and train generalizable property predictors able to extrapolate to OOD molecules.

bioinformatics↗

bioRxiv2025

Fleming: An AI Agent for Antibiotic Discovery in Mycobacterium Tuberculosis

Antibiotic development is challenged by high costs and failure rates. Artificial intelligence (AI) holds promise to overcome these challenges by predicting inhibitory properties of novel compounds, generating new candidates, and contextualizing property predictions in the biological background. Fleming is an integrative AI agent that explores novel chemical space to identify lead compounds meeting multiple criteria. The discriminative and generative AI models for Mycobacterium tuberculosis (Mtb) inhibition were trained on a set of 114,900 diverse compounds and fragments based on in vitro growth inhibition. We combined both models as well as molecular optimization, ADMET prediction and literature search functions to make Fleming an integrated agent for Mtb preclinical lead identification. Fleming has 17% higher discrimination between known Mtb leads and leads for other diseases than a generic LLM agent along with 13% higher discrimination than molecular property prediction alone on challenging ADMET tasks. Fleming demonstrates an 83% in vitro hit rate of predicted inhibition and a 100% hit rate of de novo generative design. Flemings generative designs also demonstrate an 83% rate of favorable ADMET profiles. Fleming is an integrative AI agent able to explore new regions of the chemical space to select lead compounds that simultaneously meet several desirable criteria.

bioinformatics↗

bioRxiv2025

Phyla: Towards a Foundation Model for Phylogenetic Inference

Protein language models (PLMs) are often assumed to capture evolutionary information by training on large protein sequence datasets. Yet it remains unclear whether PLMs can reason about evolution--that is, infer evolutionary relationships between sequences. We test this capability by evaluating whether standard PLM usage, frozen or fine-tuned embeddings with distance-based comparison, supports evolutionary reasoning. Existing PLMs consistently fail to recover phylogenetic structure, despite strong performance on sequence-level tasks such as masked-token and contact prediction. We present PO_SCPLOWHYLAC_SCPLOW, a hybrid state-space and transformer model that jointly processes multiple sequences and is trained using a tree-based objective across 3,000 phylogenies spanning diverse protein families. PO_SCPLOWHYLAC_SCPLOW outperforms the next-best PLM by 9% on tree reconstruction and 23% on taxonomic clustering while remaining alignment- and guide-tree-free. Although classical alignment pipelines achieve higher absolute accuracy, PO_SCPLOWHYLAC_SCPLOW narrows the gap and achieves markedly lower end-to-end runtime. Applied to real data, PO_SCPLOWHYLAC_SCPLOW reconstructs biologically accurate clades in the tree of life and resolves genome-scale relationships among Mycobacterium tuberculosis isolates. These findings suggest that, under standard usage, evolutionary reasoning does not reliably emerge from large-scale sequence modeling. Instead, PO_SCPLOWHYLAC_SCPLOW shows that models trained with phylogenetic supervision can reason about evolution more effectively, offering a biologically grounded path toward evolutionary foundation models.

bioinformatics↗

bioRxiv2024

ProCyon: A multimodal foundation model for protein phenotypes

Characterizing human proteins remains a major challenge: approximately 29% of human proteins lack experimentally validated functions and even well-annotated proteins often lack context-specific phenotypic insights. To enable universal modeling of protein phenotypes, we present PO_SCPLOWROC_SCPLOWCO_SCPLOWYONC_SCPLOW, a multimodal foundation model that utilizes protein sequence, structure, and natural language for generating and predicting protein phenotypes across diverse knowledge domains. PO_SCPLOWROC_SCPLOWCO_SCPLOWYONC_SCPLOW is trained on our novel dataset, PO_SCPLOWROC_SCPLOWCO_SCPLOWYONC_SCPLOW-IO_SCPLOWNSTRUCTC_SCPLOW, with 33 million protein phenotype instructions. On dozens of benchmarking tasks, PO_SCPLOWROC_SCPLOWCO_SCPLOWYONC_SCPLOW performs competitively against single-modal and multimodal models. Further, PO_SCPLOWROC_SCPLOWCO_SCPLOWYONC_SCPLOW conditionally retrieves proteins via mechanisms of action of small molecule drugs and disease contexts, and it generates candidate phenotypic descriptions for poorly characterized proteins, including those implicated in Parkinsons disease that were identified after PO_SCPLOWROC_SCPLOWCO_SCPLOWYONC_SCPLOWs knowledge cutoff date. We experimentally confirm PO_SCPLOWROC_SCPLOWCO_SCPLOWYONC_SCPLOWs predictions in multiple sclerosis using post-mortem brain RNA-seq, identifying novel MS genes and elucidating associated pathway mechanisms consistent with cortical pathology. PO_SCPLOWROC_SCPLOWCO_SCPLOWYONC_SCPLOW paves the way toward a general approach to generate functional insights into the human proteome.

bioinformatics↗

bioRxiv2024

Systematic review and meta-analysis of protocols and yield of direct from sputum sequencing of Mycobacterium tuberculosis

Direct sputum whole genome sequencing (dsWGS) can revolutionize Mycobacterium tuberculosis (Mtb) diagnosis by enabling rapid detection of drug resistance and strain diversity without the biohazard of culture. We searched PubMed, Web of Science and Google scholar, and identified 8 studies that met inclusion criteria for testing protocols for dsWGS. Utilising meta-regression we identify several key factors positively associated with dsWGS success, including higher Mtb bacillary load, mechanical disruption, and enzymatic/chemical lysis. Specifically, smear grades of 3+ (OR = 14.7, 95% CI: 3.5, 62.1; p = 0.0005) were strongly associated with improved outcomes, whereas decontamination with sodium hydroxide (NaOH) was negatively associated (OR = 0.005, 95% CI: 0.001, 0.03; p = 7e-06), likely due to its harsh effects on Mtb cells. Furthermore, mechanical lysis (OR = 193.3, 95% CI: 11.7, 3197.8; p = 0.008) and enzymatic/chemical lysis (OR = 18.5, 95% CI: 1.9, 183.1; p = 0.02) were also strongly associated with improved dsWGS. Across the studies, we observed a high degree of variability in approaches to sputum pre-processing prior to dsWGS highlighting the need for standardized best practices. In particular we conclude that optimizing pre-processing steps including decontamination with the exploration of alternatives to NaOH to better preserve Mtb cells and DNA, and best practices for cell lysis during DNA extraction as priorities. Further and considering the strong association between Mtb load and successful dsWGS, protocol improvements for optimal sputum sample collection, handling, and storage could also further enhance the success rate of dsWGS.

microbiology↗

bioRxiv2024

PertEval-scFM: Benchmarking Single-Cell Foundation Models for Perturbation Effect Prediction

In silico modeling of transcriptional responses to perturbations is crucial for advancing our understanding of cellular processes and disease mechanisms. We present PertEval-scFM, a standardized framework designed to evaluate models for perturbation effect prediction. We apply PertEval-scFM to benchmark zero-shot single-cell foundation model (scFM) embeddings against base-line models to assess whether these contextualized representations enhance perturbation effect prediction. Our results show that scFM embeddings do not provide consistent improvements over baseline models, especially under distribution shift. Overall, this study provides a systematic evaluation of zero-shot scFM embeddings for perturbation effect prediction, highlighting the challenges of this task and revealing the limitations of current-generation scFMs. Our findings underscore the need for specialized models and high-quality datasets that capture a broader range of cellular states. Source code and documentation can be found at: https://github.com/aaronwtr/PertEval.

bioinformatics↗

bioRxiv2024

Evaluating generalizability of artificial intelligence models for molecular datasets

Deep learning has made rapid advances in modeling molecular sequencing data. Despite achieving high performance on benchmarks, it remains unclear to what extent deep learning models learn general principles and generalize to previously unseen sequences. Benchmarks traditionally interrogate model generalizability by generating metadata based (MB) or sequence-similarity based (SB) train and test splits of input data before assessing model performance. Here, we show that this approach mischaracterizes model generalizability by failing to consider the full spectrum of cross-split overlap, i.e., similarity between train and test splits. We introduce SPECTRA, a spectral framework for comprehensive model evaluation. For a given model and input data, SPECTRA plots model performance as a function of decreasing cross-split overlap and reports the area under this curve as a measure of generalizability. We apply SPECTRA to 18 sequencing datasets with associated phenotypes ranging from antibiotic resistance in tuberculosis to protein-ligand binding to evaluate the generalizability of 19 state-of-the-art deep learning models, including large language models, graph neural networks, diffusion models, and convolutional neural networks. We show that SB and MB splits provide an incomplete assessment of model generalizability. With SPECTRA, we find as cross-split overlap decreases, deep learning models consistently exhibit a reduction in performance in a task- and model-dependent manner. Although no model consistently achieved the highest performance across all tasks, we show that deep learning models can generalize to previously unseen sequences on specific tasks. SPECTRA paves the way toward a better understanding of how foundation models generalize in biology.

bioinformatics↗

bioRxiv2019

Tuberculosis resistance acquisition in space and time: an analysis of globally diverse M. tuberculosis whole genome sequences

BackgroundMycobacterium tuberculosis (MTB) whole genome sequencing data can provide insights into temporal and geographic trends in resistance acquisition and inform public health interventions.\n\nMethodsWe curated a set of clinical MTB isolates with high quality sequencing and culture-based drug susceptibility data spanning four lineages and more than 20 countries. We constructed geographic and lineage specific MTB phylogenies and used Bayesian molecular dating to infer the most-recent-common-susceptible-ancestor age for 4,869 instances of resistance to 10 drugs.\n\nFindingsOf 8,550 isolates curated, 6,099 from 15 countries met criteria for molecular dating. The number of independent resistance acquisition events was lower than the number of resistant isolates across all countries, suggesting ongoing transmission of drug resistance. Ancestral age distributions supported the presence of old resistance, [≥]20 years prior, in the majority of countries. A consistent order of resistance acquisition was observed globally starting with resistance to isoniazid, but resistance ancestral age varied by country. We found a direct correlation between country wealth and resistance age (R2= 0.47, P-value= 0.014). Amplification of fluoroquinolone and second-line injectable resistance among multidrug-resistant isolates is estimated to have occurred very recently (median ancestral age 4.7 years IQR 1.9-9.8 prior to sample collection). We found the sensitivity of commercial molecular diagnostics for second-line resistance to vary significantly by country (P-value <0.0003)\n\nInterpretationOur results highlight that both resistance transmission and amplification are contributing to disease burden globally but are variable by country. The observation that wealthier nations are more likely to have old resistance suggests that programmatic improvements can reduce resistance amplification, but that fit resistant strains can circulate for decades subsequently.\n\nFundingThis work was supported by the NIH BD2K grant K01 ES026835, a Harvard Institute of Global Health Burke Fellowship (MF), Boston Childrens Hospital OFD/BTREC/CTREC Faculty Career Development Fellowship and Bushrod H. Campbell and Adah F. Hall Charity Fund/Charles A. King Trust Postdoctoral Fellowship (AD).\n\nResearch in contextO_ST_ABSEvidence before this studyC_ST_ABSAcquisition and spread of drug-resistance by Mycobacterium tuberculosis (MTB) varies across countries. Local factors driving evolution of drug resistance in MTB are not well studied.\n\nAdded value of this studyWe applied molecular dating to 6,099 global MTB patient isolates and found the order of resistance acquisition to be consistent across the countries examined, i.e. acquisition of isoniazid resistance first followed by rifampicin and streptomycin followed by resistance to other drugs. In all countries with data available there was evidence for transmission of resistant strains from patient-to-patient and in the majority for extended periods of time (>20 years).\n\nCountries with lower gross wealth indices were found to have more recent resistance acquisition to the drug rifampicin. Based on the resistance patterns identified in our study we estimate that commercial diagnostic tests vary considerably in sensitivity for second-line resistance diagnosis by country.\n\nImplications of all available evidenceThe longevity of resistant MTB in many parts of the world emphasizes its fitness for transmission and its continued threat to public health. The association between country wealth and recent resistance acquisition emphasizes the need for continued investment in TB care delivery and surveillance programs. Geographically relevant diagnostics that take into account a countrys unique distribution of resistance are necessary.

bioinformatics↗