Mini-batching ecological data to improve ecosystem models with machine learning
Process-based, dynamic models are essential for extrapolating beyond current trends and anticipating biodiversity responses to global change. However, their practical adoption for forecasting purposes remains limited due to difficulties in calibrating them against data and structural inaccuracies in their mathematical formulations. While ecological time series could, in principle, be used to directly estimate model parameters and refine model structures, the large noise levels in ecological datasets and the strong nonlinearity of ecological dynamics challenge conventional calibration methods. Here, we present a robust and scalable calibration framework that addresses these challenges by integrating techniques from scientific computing and deep learning. Our approach combines a segmentation strategy where state variables are estimated independently, differentiable programming for efficient gradient computation, parameter transformations to ensure the feasibility and stability of the model simulations, and mini-batching to accomodate large datasets. Through comprehensive benchmarks using simulated food web dynamics of increasing complexity, we demonstrate that the framework substantially improves the convergence of gradient descent algorithms and Monte Carlo sampling methods, accommodating for realistic scenarios with noisy and partial observations. This yields improved parameter estimation and forecasts within both mode estimation and full posterior distribution contexts. Crucially, we show that the calibration framework scales effectively with both the number of parameters and state variables. The improved convergence and scalability of the calibration framework enables hybrid modeling approaches, where neural networks parameterize complex processes within the dynamic model. In particular, we demonstrate that neural networks can effectively capture environmental dependencies in demographic rates and recover functional responses governing trophic interactions. Neural network-based parameterizations have the capability to improve the structural inaccuracies of models while maintaining ecological interpretability through post-hoc analysis of learned representations. We provide an implementation of the calibration framework and other key utilities as the open-source Julia package HybridDynamicModels.jl, with the hope that the package will facilitate the development of hybrid modeling approaches in quantitative ecology and related fields.