Reward prediction errors induce risk-seeking

Abstract

Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. The common neural substrate suggests that RPEs and risk preferences might be linked on the level of behavior as well, but this has never been tested. Here, we aim to close this gap. First, we apply a recent theory of learning in the basal ganglia to predict how exactly RPEs might influence risk preferences. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that subjects become more risk seeking if choices are preceded by positive RPEs, and more risk averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates. Finally, we show that RPE-induced risk-seeking is indexed by pupil dilation: participants with stronger pupillary correlates of RPE also show more pronounced behavioral effects. Authors summaryMany of our decisions are based on expectations. Sometimes, however, surprises happen: outcomes are not as expected. Such discrepancies between expectations and actual outcomes are called prediction errors. Our brain recognises and uses such prediction errors to modify our expectations and make them more realistic--a process known as reinforcement learning. In particular, neurons that release the neurotransmitter dopamine show activity patterns that strongly resemble prediction errors. Interestingly, the same neurotransmitter is also known to regulate risk preferences: dopamine levels control our willingness to take risks. We theorised that, since learning signals cause dopamine release, they might change risk preferences as well. In this study, we test this hypothesis. We find that participants are more likely to make a risky choice just after they experienced an outcome that was better than expected, which is precisely what out theory predicts. This suggests that dopamine signalling can be ambiguous--a learning signal can be mistaken for an impulse to take a risk.