Reproducibility and model-selection stability in connectome-constrained circuit modeling
Connectome-constrained neural network models aim to link anatomical connectivity with functional computation by training networks whose architectures reflect biological circuits. Because such models are increasingly used to infer neural mechanisms, it is important to assess their robustness to variations in training conditions and model selection criteria. Here we retrain ensembles of connectome-constrained models under nominally identical conditions and compare their correspondence to experimentally measured response properties in the Drosophila motion pathway. While task performance remains similar across models, the identification of biologically plausible circuit solutions is unstable across retraining runs. In particular, model clusters selected by lowest validation task error do not reliably correspond to experimentally observed neural tuning, and small variations in performance metrics can reorder cluster rankings. These results indicate that, in this framework, similar task performance does not reliably identify biologically plausible circuit solutions. Task error alone is therefore insufficient for mechanistic identification, and additional model-selection criteria are needed.