bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.64898/2026.10.01.752251

VINTER: a generative vision-language model for annotating and interrogating spatial tissues

Wu, J.·Xu, W.·Zhuang, Y.·Zhang, Y.·Loza Lopez, M. d. J.·yang, y.·Nakai, K.
View original source ↗Export metadata ↓

Abstract

Spatial transcriptomics links gene expression to tissue structure, yet current models learn embeddings whose meaning is assigned afterwards or read histology without measured expression. Tissue decisions are therefore rarely made from morphology, genes and neighbourhood together, or examined through them. Here we introduce VINTER, a generative vision-language model aligned on 1.45 million histology-caption pairs that reads histology, measured gene expression and spatial neighbourhood as one sequence. It returns the probability of every tissue class, and because genes enter as text, its molecular evidence can be edited with the image held fixed. Reading all three sources together, VINTER annotated tissue more accurately than spatial-domain methods and a spatial foundation model, and transferred across cohorts without target labels. Editing only the neighbourhood resolved the follicular B-cell and plasma-cell programmes behind its reading of tertiary lymphoid structure (TLS) maturation in lung and kidney cancer. Its class probabilities exposed what single labels conceal in triple-negative breast cancer: immune heterogeneity within stroma and cancer hidden beneath benign labels. Without retraining, the same model transferred to Visium HD and single-cell Xenium data and resolved follicular architecture within colorectal cancer TLS. VINTER thus turns spatial annotation into an interrogable reading of tissue across platforms and resolutions.

Explore related subjects

bioinformaticsPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Wu, J., Xu, W., Zhuang, Y., Zhang, Y., Loza Lopez, M. d. J., yang, y., Nakai, K.. 2026-10-08. VINTER: a generative vision-language model for annotating and interrogating spatial tissues. https://doi.org/10.64898/2026.10.01.752251

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

EMBER: Contextualized gene representations enable compositional prediction across cellular perturbations

Predicting the effects of individual and combined perturbations is central to identifying context-specific vulnerabilities and designing targeted interventions. Yet learning gene representations that integrate functional relationships with cellular state and transfer across perturbation tasks remains challenging. Here, we present EMBER, a gene-centric foundation model that learns context-dependent functional representations of genes. EMBER combines interaction-guided contrastive learning with masked expression reconstruction during pretraining on a single-cell corpus curated for diversity and balanced sampling across cell types and experimental batches. EMBER decouples gene representation learning from task-specific prediction through a composable embedding framework: the pretrained encoder remains frozen, while lightweight probes combine contextualized gene embeddings to predict individual and combined perturbation effects. The same pretrained representations support single-gene perturbation-response prediction, digenic interaction prediction, and multi-target drug-response prediction. EMBER improves cross-context perturbation-response prediction and genetic-interaction ranking relative to evaluated baselines. EMBER also prioritized SCFD1, a mediator of autophagosome-lysosome fusion, as a candidate dependency during mTOR inhibition. Together, these results establish contextualized gene embeddings as reusable building blocks for modeling genetic and pharmacological perturbations, supporting the prioritization of genetic interactions and context-specific interventions for experimental testing.

bioinformatics↗
bioRxiv2026

Small molecule structure elucidation from NMR spectra with generative and reasoning models

Elucidating the structures of unknown small molecules remains a major challenge in chemistry, pharmacology, and drug discovery. While nuclear magnetic resonance (NMR) spectroscopy is the gold standard for structure elucidation, interpreting spectra remains labor-intensive and requires expert knowledge to navigate the vast space of possible structures. Here, we present an AI system for rapid, automated de novo structure elucidation that combines a molecular generative model with a scientific reasoning agent. Its generative component, ChefNMR2, is an atomic flow-matching model that predicts small-molecule structures from molecular formulae and one-dimensional 1H and 13C NMR spectra. ChefNMR2-Agent augments our generative model with large language models (LLMs) to reason over and refine candidate structures. Across diverse benchmarks, ChefNMR2 substantially outperforms existing tools, offering calibrated confidence estimates and two to three orders of magnitude faster inference than frontier LLMs. We use ChefNMR2-Agent to elucidate the structures of four novel natural products, matching human-chemist accuracy in a fraction of the time. Our work demonstrates how specialized generative models and general-purpose reasoning agents can unite to solve challenging scientific inverse problems and accelerate NMR structure elucidation, thus opening new possibilities for AI-driven small-molecule discovery across diverse disciplines.

bioinformatics↗
bioRxiv2026

Digital twins forecast individual immune trajectories

The human immune system differs markedly between individuals and changes in response to infection, vaccination and treatment. AI virtual cells are being built to simulate cellular responses to perturbation. Yet these models, like most immune-response predictors, describe each person by a single pre-challenge measurement and forecast change from patterns pooled across people. Digital twins, updated with every new observation of an individual, could forecast how that person's immune state changes. Here we present ImmuneTwins, which builds these twins from single-cell profiles by personalizing a shared virtual-cell model with each individual's observations. In two external populations, ImmuneTwins reduced immune-age error by 32% and 42% relative to the best existing method. During repeated influenza vaccination, its twins predicted the direction of immune-cell change 35% more accurately than the best published method, and their error fell by 27% as observations accumulated. The twins revealed that immune cells and plasma proteins recover about half their deviation from a persistent personal reference within 90 days, whereas response size varies between challenges. They further localized B-cell remodelling in exceptional longevity and anticipated patient-specific immune trajectories during myeloma treatment. These results establish virtual-cell digital twins for forecasting individual immune change and identifying each person's characteristic immune features.

bioinformatics↗