An Optimal Policy Model for Concurrent Uncertainty Estimation During Decision Making

Abstract

We often postpone or even avoid making decisions when we feel uncertain. Uncertainty estimation is not an afterthought of decision making but a dynamic process that accompanies decision making in parallel and affects decision making. To study concurrent uncertainty estimation during decision making, we adapted the classic random-dots motion direction discrimination task to allow a reaction-time measure of uncertainty responses. Subjects were asked to judge whether a patch of random dots was moving left or right. In addition, they could seek assistance by choosing to look at a second stimulus that had the same direction but high coherence any time during the task. The task allows us to measure the reaction time of both the perceptual decisions and the uncertainty responses. The subjects were more likely to choose the uncertainty response when the motion coherence was low, while their reaction times of the uncertainty responses showed individual variations. To account for the subjects behavior, we created an optimal policy decision model in which decisions are based on the value functions computed from the accumulated evidence using a drift-diffusion process. Model simulations captured key features of the subjects choices, reaction times, and proportions of uncertainty responses. Varying model parameters explained individual variations in the subjects and the correlations between decision accuracy, proportions of uncertainty responses, and reaction times at the individual level. Our model links perceptual decisions and value-based decisions and indicates that concurrent uncertainty estimation may be based on comparisons between values of uncertainty responses and perceptual decisions, both of which may be derived from the same evidence accumulation process during decision making. It provides a theoretical framework for future investigations, including the ones that aim at the underlying neural mechanism.