Cross-session generalization in automated behavioral tracking of Galleria mellonella larvae: comparison of classical computer vision, deep learning and generative domain adaptation

Abstract

BackgroundAutomated behavioral tracking is increasingly used in biological and biomedical research; however, robustness across heterogeneous imaging conditions remains a major challenge. Domain shifts caused by changes in illumination, contrast, or acquisition setup can substantially degrade the performance of computer vision and deep learning models, limiting their practical applicability. This issue is particularly relevant for small-scale biological datasets, where extensive annotation and retraining are often impractical. MethodsWe developed a behavioral tracking framework for Galleria mellonella larvae combining a classical computer vision (CV) pipeline, a YOLOv8s-seg + ByteTrack deep learning pipeline, and generative domain adaptation methods. Behavioral recordings were collected in two independent experimental sessions under distinct illumination conditions (top and bottom lighting), creating a natural cross-session domain shift. The classical pipeline was based on contour detection, adaptive preprocessing, temporal smoothing, and trajectory reconstruction. Deep learning models were trained on 320 manually annotated frames and evaluated in both within-session and cross-session settings. To improve generalization, we investigated generative augmentation using StyleGAN2-ADA and unpaired image-to-image translation using CycleGAN. Tracking outputs were further analyzed through behavioral descriptors, including trajectories, traveled distance, velocity, spatial occupancy heatmaps, and directional movement patterns. ResultsThe classical CV pipeline achieved high detection performance in both recording sessions, with mean detection rates of 99.05% and 99.71%, respectively. A YOLOv8s-seg model demonstrated strong within-session performance but exhibited severe degradation under cross-session evaluation, with larval mask segmentation performance dropping to mAP@0.5 = 9.1%, confirming the presence of a substantial domain shift. Despite differences in detection methodology, behavioral metrics derived from YOLO and CV pipelines showed strong agreement at the group level (Pearson correlation r = 0.89; median distance ratio = 0.99). Generative augmentation with StyleGAN2-ADA did not yield meaningful gains in tracking robustness -- likely because baseline performance was already near ceiling -- whereas CycleGAN-based domain adaptation substantially reduced the domain gap and improved cross-session detection performance while preserving biologically relevant trajectory structures and spatial behavioral patterns. ConclusionsCross-session variability represents a critical challenge for automated behavioral tracking in biological experiments. Our results demonstrate that carefully designed classical computer vision approaches can achieve highly reliable tracking in small-data settings, while deep learning models require explicit strategies to address domain shift. Generative domain adaptation, particularly CycleGAN-based image translation, offers an effective solution for improving cross-session generalization without additional manual annotation. The proposed framework provides a robust foundation for scalable behavioral phenotyping of Galleria mellonella and other small biological model organisms.