JEPA-SORT: World-Model Embeddings Improve Deep-Sea Multi-Object Tracking on Visually Homogeneous Fauna
Protecting deep-sea sessile communities as Marine Protected Areas (MPAs) requires accurate estimates of their spatial distribution and abundance. Existing computer-vision detection methods require careful calibration to avoid counting errors; multi-object tracking (MOT) theoretically offers a more robust alternative. In practice, however, poor lighting and turbidity limit the efficacy of purely kinematic MOT algorithms in deep-sea environments. At the same time, the intra-species visual homogeneity typical of deep-sea scenes means traditional per-frame appearance features are often too weak to maintain stable identities in deep-association MOT algorithms. In this paper we introduce JEPA-SORT, an MOT algorithm that injects V-JEPA 2.1 embeddings into detection-to-track association, to show that video world-model embeddings - which capture not only physical appearance but spatiotemporal trajectory across viewpoints - can help maintain stable identities on visually homogeneous specimens. We provide a benchmark of twelve ROV transects surveying deep-sea sessile communities and evaluate JEPA-SORT against standard MOT algorithms across a range of simulated optical environments. We also propose the Biodensity Utility Metric (BDU), which measures tracker performance in the context of downstream ecological modeling. JEPA-SORT outperforms all benchmarked trackers on IDF1, MOTA, and BDU and is competitive on specimen-count accuracy. Ablations indicate its advantage stems primarily from the discriminative power of world-model embeddings on visually homogeneous footage.