A Theoretical Formalization of Consequence-Based Decision-Making

Abstract

Learning to make adaptive decisions depends on exploring options, experiencing their consequence, and reassessing ones strategy for the future. Although several studies have analyzed various aspects of value-based decision-making, most of them have focused on decisions in which gratification is cued and immediate. By contrast, how the brain gauges delayed consequence for decision-making remains poorly understood. To investigate this, we designed a decision-making task in which each decision altered future options. The task was organized in groups of consecutively dependent trials, and the participants were instructed to maximize the cumulative reward value within each group. In the absence of any explicit performance feedback, the participants had to test and internally assess specific criteria to make decisions. This task was designed to specifically study how the assessment of consequence forms and influences decisions as learning progresses. We analyzed behavior results to characterize individual differences in reaction times, decision strategies, and learning rates. We formalized this operation mathematically by means of a multi-layered decision-making model. By using a mean-field approximation, the first layer of the model described the dynamics of two populations of neurons which characterized the binary decision-making process. The other two layers modulated the decision-making policy by dynamically adapting an oversight learning mechanism. The model was validated by fitting each individual participants behavior and it faithfully predicted non-trivial patterns of decision-making, regardless of performance level. These findings provided an explanation to how delayed consequence may be computed and incorporated into the neural dynamics of decision-making, and to how learning occurs in the absence of explicit feedback.