bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.64898/2026.10.07.757484

JEPA-SORT: World-Model Embeddings Improve Deep-Sea Multi-Object Tracking on Visually Homogeneous Fauna

Tillinghast-Raby, A.·Kravetz, M.·Kamau-Weng, J.·Weinnig, A.·Rodrigues, Y.·Varini, F.·Pieribone, V.
View original source ↗Export metadata ↓

Abstract

Protecting deep-sea sessile communities as Marine Protected Areas (MPAs) requires accurate estimates of their spatial distribution and abundance. Existing computer-vision detection methods require careful calibration to avoid counting errors; multi-object tracking (MOT) theoretically offers a more robust alternative. In practice, however, poor lighting and turbidity limit the efficacy of purely kinematic MOT algorithms in deep-sea environments. At the same time, the intra-species visual homogeneity typical of deep-sea scenes means traditional per-frame appearance features are often too weak to maintain stable identities in deep-association MOT algorithms. In this paper we introduce JEPA-SORT, an MOT algorithm that injects V-JEPA 2.1 embeddings into detection-to-track association, to show that video world-model embeddings - which capture not only physical appearance but spatiotemporal trajectory across viewpoints - can help maintain stable identities on visually homogeneous specimens. We provide a benchmark of twelve ROV transects surveying deep-sea sessile communities and evaluate JEPA-SORT against standard MOT algorithms across a range of simulated optical environments. We also propose the Biodensity Utility Metric (BDU), which measures tracker performance in the context of downstream ecological modeling. JEPA-SORT outperforms all benchmarked trackers on IDF1, MOTA, and BDU and is competitive on specimen-count accuracy. Ablations indicate its advantage stems primarily from the discriminative power of world-model embeddings on visually homogeneous footage.

Explore related subjects

ecologyPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Tillinghast-Raby, A., Kravetz, M., Kamau-Weng, J., Weinnig, A., Rodrigues, Y., Varini, F., Pieribone, V.. 2026-10-09. JEPA-SORT: World-Model Embeddings Improve Deep-Sea Multi-Object Tracking on Visually Homogeneous Fauna. https://doi.org/10.64898/2026.10.07.757484

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

On Motivating the Choice of Link Function in Binomial Regression Through A Latent Abundance Process

Ecologists commonly apply binomial regression to study binary data. Often, the link function in binomial regression is not purposefully chosen, but instead selected on the grounds of interpretation or computation efficiency. However, if the binary response is the result of a latent abundance process, then the link function implies assumptions for this underlying process. The most well-known example of this is the complementary log-log (cloglog) link, which has direct connections with the probability of a positive count under a latent Poisson distribution. If the binary response can be assumed to represent a truncated latent count, the cloglog link can be a better choice than the logit link. On the other hand, counts in ecology are often overdispersed, meaning the assumption of an underlying Poisson abundance process is unrealistic. In this article, we provide a framework for consideration of the link choice in binomial regression, in terms of the latent abundance process that binary responses can represent. Aside from the Poisson case mentioned above, we show that the cloglog link is the right choice if the latent abundance process represents the biomass of individuals, or if the latent abundance process comes from an overdispersed count distribution with a linear mean-variance relationship. For the case where the latent count follows a negative binomial distribution with a quadratic mean-variance relationship, we derive a new generalized complementary log-log (gcloglog) link function, which includes the case of the cloglog link as well as that of the logit link, where the latter implies assuming moderate overdispersion in the form of a geometric distribution. Two simulation studies demonstrate that cloglog and logit link binomial regression can exhibit bias and undercoverage when the latent count is more overdispersed than assumed, while gcloglog link models overcome this issue but in practice require replication in the binomial response for reliable estimation. Two real ecological datasets further illustrate how to use the gcloglog link binomial regression in practice including as a means of sensitivity analysis. We encourage ecologists to give more thought to the choice of link function in binomial regression, based on considering the latent abundance process and its properties.

ecology↗
bioRxiv2026

Towards predictive ecological biogeography: Predicting vegetation type using fine-scale measurements of the abiotic environment

One of the goals of ecological biogeography is to accurately predict where major vegetation types, such as forests or grasslands, can establish themselves. Alas, our current understanding of the mechanisms by which environmental variables jointly influence vegetation types does not allow for such predictions. Here, we compile a database of local measurements of soil, weather, topography, and bedrock lithology for 6404 sites across the continental United States, which we complement with information about the vegetation around the measurement sites. By applying machine learning methods to a subset of the sites, we are able to train a machine that predicts - using only information about abiotic variables - the presence of forests or grasslands with nearly 90% accuracy on the rest of the sites. We find that while weather variables are statistically important for these predictions, a few soil variables prove to be the most powerful predictors. By separating the combined contributions of weather and soil, we show that soil texture is a better predictor under dry conditions than under wet conditions, while pH is important predictor regardless of the weather. Furthermore, from 136 sites undergoing grassland-to-forest succession, our analysis indicates that soil pH changes with a certain lag relative to vegetation change. Sites that have recently undergone a transition exhibit a higher pH, closer to that of grasslands, despite the presence of trees. The pH then decreases by about one unit over a 30-year period of forest development. Ultimately, our results indicate that fine-scale environmental measurements and historical ecological data can serve as a basis for quantitative bio-geographical predictions, which could eventually guide targeted conservation interventions.

ecology↗
bioRxiv2026

Invasive Chinese tallow is associated with distinct rhizosphere microbial communities, core microbiomes, and functional potential relative to native Loblolly pine

Plant invasions frequently reshape belowground microbial communities, yet their effects on rhizosphere microbiome structure and function remain poorly understood. We compared the rhizosphere bacterial and fungal microbiomes of invasive Chinese tallow (Triadica sebifera) and native Loblolly pine (Pinus taeda) using 16S rRNA gene and ITS amplicon sequencing, together with microbial, functional, and soil analyses. Chinese tallow exhibited significantly greater bacterial and fungal richness and distinct microbial community composition than Loblolly pine. The invasive rhizosphere was enriched in plant-beneficial bacterial taxa, including Streptomyces, Bradyrhizobium, and Rhizobium, and ectomycorrhizal-associated fungi such as Russula and Cenococcum. This enrichment is consistent with enhanced nutrient acquisition, plant growth promotion, and stress tolerance, potentially favoring Chinese tallow establishment and persistence. Further, core microbiome and co-occurrence network analyses revealed extensive reorganization of persistent (Streptomyces, Russula) and highly connected microbial taxa that may be associated with Chinese tallow invasion. Functional predictions indicated greater representation of carbohydrate degradation, nitrogen fixation, plant growth promotion, nutrient mineralization, and ectomycorrhizal guilds in the Chinese tallow rhizosphere, whereas lignin degradation and saprotrophic or pathogen-associated fungal guilds were relatively more abundant under Loblolly pine. Soil physicochemical properties were also strongly correlated with dominant bacterial and fungal taxa, indicating that host-mediated microbial recruitment and environmental filtering jointly shaped rhizosphere community assembly. Overall, our findings identify rhizosphere microbiome reassembly as an important belowground feature associated with Chinese tallow invasion.

ecology↗