Animal learning in amultidimensional discrimination task as explained by dimension-specific allocation of attention

Abstract

Reinforcement learning describes the process by which during a series of trial-and-error attempts, actions that culminate in reward are strengthened. When the actions are based on sensory stimuli, an association is formed between the stimulus, the action and the reward. Computational, behavioral and neurobiological accounts of this process successfully explain simple stimulus-response learning. However, if the cue is multi-dimensional, identifying which of its features are relevant for the reward is not trivial, and the underlying cognitive process is poorly understood. To study this we adapted an intra-dimensional/ extra-dimensional set-shifting paradigm to train rodents on a multidimensional sensory discrimination task. In our setup, stimuli of different modalities (spatial, olfactory and visual) are combined into complex cues and manipulated independently. In each set, only a single stimulus dimension is relevant for reward. To distinguish between learning and decision-making we suggest a weighted attention model (WAM). It combines a learning model where each feature-dimension is reinforced separately with a decision rule that chooses an alternative according to a weighted average of learnt values, in which weight is associated with each dimension. We estimated the parameters of the WAM (decision weights, learning rate and noise) and demonstrated that is outperforms an alternative model in which a value learnt is assigned to each combination of features, or every state. Estimated decision weights of WAM reveal an experience-based bias in learning. The intra-dimensional set shift separated the decision weights. While in the first phase of the experiment the weights were roughly the same, in the second phase the weight on the dimension that was key to finding the reward became higher than others. After the extra-dimensional shift this dimension became irrelevant, however its decision weight remained high for the early learning stage in this last phase, providing an explanation for the poor performance of the animals. By the end of the phase when the rats performance improved, the weights for the two dimensions converged. Thus, estimated weights can be viewed as a possible way to quantify the experience-based bias.