The shadowing effect of initial expectation on learning asymmetry

Abstract

Evidence for positivity and optimism bias abounds in high-level belief updates. However, no consensus has been reached regarding whether learning asymmetries exists in more elementary forms of updates such as reinforcement learning (RL). In RL, the learning asymmetry concerns the sensitivity difference in incorporating positive and negative prediction errors (PE) into value estimation, namely the asymmetry of learning rates associated with positive and negative PEs. Although RL has been established as a canonical framework in interpreting agent and environment interactions, the direction of the learning rate asymmetry remains controversial. Here, we propose that part of the controversy stems from the fact that people may have different value expectations before entering the learning environment. Such default value expectation influences how PEs are calculated and consequently biases subjects choices. We test this hypothesis in two learning experiments with stable or varying reinforcement probabilities, across monetary gains, losses and gain-loss mixtures environments. Our results consistently support the model incorporating asymmetric learning rates and initial value expectation, highlighting the role of initial expectation in value update and choice preference. Further simulation and model parameter recovery analyses confirm the unique contribution of initial value expectation in accessing learning rate asymmetry. Author SummaryWhile RL model has long been applied in modeling learning behavior, where value update stands in the core of the learning process, it remains controversial whether and how learning is biased when updating from positive and negative PEs. Here, through model comparison, simulation and recovery analyses, we show that accurate identification of learning asymmetry is contingent on taking into account of subjects default value expectation in both monetary gain and loss environments. Our results stress the importance of initial expectation specification, especially in studies investigating learning asymmetry.