Nariman Zadeh, H. | bioRxiv Science

bioRxiv2020

The chaos in calibrating crop models

Calibration, the estimation of model parameters based on fitting the model to experimental data, is among the first steps in many applications of system models and has an important impact on simulated values. Here we propose and illustrate a novel method of developing guidelines for calibration of system models. Our example is calibration of the phenology component of crop models. The approach is based on a multi-model study, where all teams are provided with the same data and asked to return simulations for the same conditions. All teams are asked to document in detail their calibration approach, including choices with respect to criteria for best parameters, choice of parameters to estimate and software. Based on an analysis of the advantages and disadvantages of the various choices, we propose calibration recommendations that cover a comprehensive list of decisions and that are based on actual practices. HighlightsO_LIWe propose a new approach to deriving calibration recommendations for system models C_LIO_LIApproach is based on analyzing calibration in multi-model simulation exercises C_LIO_LIResulting recommendations are holistic and anchored in actual practice C_LIO_LIWe apply the approach to calibration of crop models used to simulate phenology C_LIO_LIRecommendations concern: objective function, parameters to estimate, software used C_LI

plant biology↗

bioRxiv2019

How well do crop models predict phenology, with emphasis on the effect of calibration?

Predicting phenology is essential for adapting varieties to different environmental conditions and for crop management. Therefore, it is important to evaluate how well different crop modeling groups can predict phenology. Multiple evaluation studies have been previously published, but it is still difficult to generalize the findings from such studies since they often test some specific aspect of extrapolation to new conditions, or do not test on data that is truly independent of the data used for calibration. In this study, we analyzed the prediction of wheat phenology in Northern France under observed weather and current management, which is a problem of practical importance for wheat management. The results of 27 modeling groups are evaluated, where modeling group encompasses model structure, i.e. the model equations, the calibration method and the values of those parameters not affected by calibration. The data for calibration and evaluation are sampled from the same target population, thus extrapolation is limited. The calibration and evaluation data have neither year nor site in common, to guarantee rigorous evaluation of prediction for new weather and sites. The best modeling groups, and also the mean and median of the simulations, have a mean absolute error (MAE) of about 3 days, which is comparable to the measurement error. Almost all models do better than using average number of days or average sum of degree days to predict phenology. On the other hand, there are important differences between modeling groups, due to model structural differences and to differences between groups using the same model structure, which emphasizes that model structure alone does not completely determine prediction accuracy. In addition to providing information for our specific environments and varieties, these results are a useful contribution to a knowledge base of how well modeling groups can predict phenology, when provided with calibration data from the target population.

plant biology↗