The dynamics of explore-exploit decisions suggest a threshold mechanism for reduced random exploration in older adults

Abstract

When faced with a choice between exploring an unknown option vs exploiting an option they know well, older adults explore less and exploit more than younger adults. Recent work has suggested that one cause of this age difference in exploration is a reduction in the extent to which older adults use "random exploration" - exploration driven by behavioral variability. Here we investigate potential mechanisms for this age-related difference in random exploration through the lens of a drift diffusion model (DDM) of the explore-exploit choice. In this model, random exploration can be modulated by two mechanisms - the fidelity with which information about the choice is represented in the brain, the "signal-to-noise ratio" (SNR), and the amount of information required to make a decision, the "decision threshold." Reduced random exploration in aging could be caused either by an increase in signal-to-noise ratio or an increase in decision threshold in older adults. By fitting the DDM to choices and response times in a sample of healthy younger and older adults, we found that older adults had a lower SNR and a higher threshold than younger adults. This suggests that reduced random exploration in aging is driven by higher response thresholds in older adults, which may compensate for the reduced signal-to-noise ratio with which decision information is represented in the brain. Author SummaryThe balance of deciding to explore the unknown, versus exploiting the well-known, changes with age. Compared to younger counterparts, healthy older adults have reduced random exploration, in which choices appear uninfluenced by the value of options. To investigate the mechanism of reduced random exploration, reaction time and choices between two slot machines in the Horizon Task were modelled using the drift diffusion model (DDM). In the DDM there is an accumulation of evidence over time, until a boundary threshold for either the explore or exploit option is crossed. The DDM used here can distinguish between two different drivers of random exploration, changes in the signal-to-noise ratio (SNR), with which reward information is represented, and changes in the threshold required to make a decision. We showed that reduced random exploration in older adults results from a higher decision threshold. Whilst older adults had a lower SNR than younger adults, which could lead to more mistakes, older adults actually performed slightly better in the Horizon task than younger adults. Together this suggests that the higher decision threshold could be a healthy aging adaptation, which is overcompensating for the less accurate choices that could result from a lower SNR alone.