Environmental dynamics impact whether matching is optimal

Abstract

When foraging for resources, animals must often sample many options that yield reward with different probabilities. In such scenarios, many animals have been shown to exhibit "matching", an empirical behavioral observation in which the fraction of rewarded samples is the same across all options. While previous work has shown that matching can be optimal in environments with diminishing returns, this condition is not sufficient to determine optimality. Furthermore, while diminishing returns naturally arise when resources in the environment deplete and take time to be replenished, the specific form of diminishing returns depends on the temporal structure and statistics of the replenishment process. Here, we explore how these environmental properties affect whether matching is optimal. By considering an agent that samples different options with fixed sampling rates, we derive the probability of collecting a reward as a function of these sampling rates for different types of environments, and we analytically determine the conditions under which the optimal sampling-rate policy exhibits matching. When all options are governed by the same replenishment dynamics, we find that optimality gives rise to matching across a wide range of environments. However, when these dynamics differ across options, the optimal policy can deviate from matching. In such cases, the rank-ordering of observed reward probabilities depends only on the qualitative nature of the replenishment process, but not on the specific replenishment rates. As a result, the optimal policy can exhibit underor over-matching depending on how rewarding the different options are. We use this result to identify environmental settings under which performance differs substantially between matching and optimality. Finally, we show how fluctuations in these replenishment rates--which can represent either environmental stochasticity or the agents internal uncertainty about the environment--can accentuate deviations between optimality and matching. Together, these findings deepen our understand of the relationship between environmental variability and behavioral optimality, and they provide testable experimental predictions across a wide range of environmental settings.