Striatal compartments participate in multi-modal and concurrent reward-based learning

Abstract

Cortical and basal ganglia circuits play a crucial role in the formation of goal-directed and habitual behaviors. In this study, we investigate the cortico-striatal circuitry involved in learning and the role of this circuitry in the emergence of inflexible behaviors such as those observed in addiction. Specifically, we develop a computational model of cortico-striatal interactions that performs concurrent goal-directed and habit learning. The model accomplishes this by distinguishing learning processes in the dorsomedial striatum (DMS) that rely on reward prediction error signals as distinct from the dorsolateral striatum (DLS) where learning is supported by salience signals. These striatal subregions each operate on unique cortical input: the DMS receives input from the prefrontal cortex (PFC) which represents outcomes, and the DLS receives input from the premotor cortex which determines action selection. Following an initial learning of a two-alternative forced choice task, we subjected the model to reversal learning, reward devaluation, and learning a punished outcome. Behavior driven by stimulus-response associations in the DLS resisted goal-directed learning of new reward feedback rules despite devaluation or punishment, indicating the expression of habit. We repeated these simulations after the impairment of executive control, which was implemented as poor outcome representation in the PFC. The degraded executive control reduced the efficacy of goal-directed learning, and stimulus-response associations in the DLS were even more resistant to the learning of new reward feedback rules. In summary, this model describes how circuits of the dorsal striatum are dynamically engaged to control behavior and how the impairment of executive control by the PFC enhances inflexible behavior. Author SummarySubstance abuse changes an animals brain so that it is not able to appropriately respond to new environmental cues such as negative consequences for drug use. It is thought that the transition from adaptive to inflexible responding involves the inappropriate engagement of neural circuitry responsible for habit. In this study, we model neural circuitry that is critical for adaptive responding to changes in reward feedback to animal behavior. We extended previous computational models of the basal ganglia to incorporate detailed biophysical representations of the dorsomedial striatum, which is critical for goal-directed behavior, and the dorsolateral striatum, which is critical for habitual behavior. The model exhibited learning in simulations of reward reversal, devaluation, and punished outcome tasks. Recent work from our group has quantified the impairment of executive control in cortical activity of compulsively drinking rats. Here, we challenged the model in additional simulations by implementing the impairment of executive control as a reduction in the fidelity of prefrontal cortical outcome representations. Our results illustrate how impaired executive control reduces the performance of goal-directed learning and, hence, reduces the models ability to overcome an established habitual response.