Biology subjects

Balleine, B. W.

Publications and source records attributed to Balleine, B. W..

17 recordsLinked to original sources

bioRxiv2026

Distinct amygdala-striatal and amygdala-cortical circuits encode and retrieve predictive learning to guide choice and decision-making

It has long been known that Pavlovian cues can influence choice between instrumental actions, although how this is achieved at a neural level is not fully understood. Here we examined the neural processes engaged during the encoding and retrieval of the Pavlovian stimulus-outcome (S-O) associations prior to and during their influence on choice. Using a Pavlovian-instrumental transfer paradigm (PIT), we found, as previously reported, that a circuit connecting the basolateral amygdala (BLA) and nucleus accumbens shell (BLA[->]NAc-S) is necessary to encode specific S-O associations within the NAc-S. However, this circuit was not required for the retrieval of those associations: optogenetic inhibition of this circuit had no effect on the ability of specific S-O associations to influence choice between instrumental actions. The BLA also projects to the infralimbic cortex (IL) although, using circuit mapping, we found that this projection was largely independent of the projection to the NAc-S. Nevertheless, using a circuit-specific disconnection approach, we found that the BLA[->]IL circuit is necessary for both the encoding and retrieval of specific S-O associations to influence choice. Furthermore, expanding on this disconnection approach, optogenetic inhibition of the BLA[->]IL circuit in one hemisphere and the IL[->]NAc-S contralaterally provided evidence that the broader BLA[->]IL[->]NAc-S circuit is necessary to retrieve specific S-O associations to influence choice during PIT. These data reveal, therefore, evidence of an amygdala-cortico-striatal circuit necessary for the retrieval of predictive learning to influence choice that differs from the circuit involved in encoding those predictive associations.

animal behavior and cognition↗

bioRxiv2026

The control of goal-directed actions by nutrient-specific appetites and rewards

There is evidence that appetites for specific nutrients can guide foraging behaviour and aid in dietary regulation through associative learning processes that link stimuli to nutrient-specific outcomes. However, most, if not all, examples of such behaviour can be interpreted as being stimulus-bound habits, i.e., reflexive responses induced by environmental stimuli. The control of identified goal-directed actions by nutrient-specific appetites has not been directly assessed. To address this question, we trained rats to press a lever for a high protein reward (whey protein shake) and another lever for a high carbohydrate reward (polycose solution). They were then tested under extinction conditions in which both levers were available following the extended exposure to meals that were high in protein or carbohydrate. When otherwise food-deprived rats had been selectively satiated on protein immediately prior to test, they pressed more on the lever they learned had produced polycose, whereas they pressed the lever they learned had produced whey protein more if they had instead been satiated on carbohydrate. Crucially, the same pattern emerged whether the satiety manipulation was achieved using the same nutrient sources that rats had earned during training (i.e., whey or polycose) or with foods high in the relevant nutrients, indicating that these behaviours were under goal-directed control and sensitive to nutritional state. These results show that actions can be motivated by the nutritional relevance of the instrumental outcome to specific appetites, a relationship that may guide natural foraging decisions.

animal behavior and cognition↗

bioRxiv2025

The influence of nucleus accumbens shell D1 and D2 neurons on outcome-specific Pavlovian instrumental transfer

The nucleus accumbens shell (NAc-S) and its projections to the ventral pallidum (VP) are thought to be critical for stimulus-based decisions. The NAc-S is predominantly composed of spiny projection neurons (SPNs) that express either the dopamine D1 (D1-SPNs) or the dopamine D2 receptor (D2-SPNs). Yet, the role of these two neuronal subpopulations and their inputs to the VP in stimulus-based decisions remains unknown. Here, we used optogenetics in female and male knock-in rats to selectively silence D1-or D2-SPNs and their projections to the VP at a time when the rats were required to use predictive stimuli to choose between two instrumental actions. Silencing either population of NAc-S SPNs disrupted choice. Silencing NAc-S D1-SPNs terminals in the VP also disrupted choice. However, choice was left intact by silencing NAc-S D2-SPNs terminals in the VP. Together, these findings provide novel insights into the cellular mechanisms and circuitry underlying stimulus-based decisions. We discuss how these insights are consistent with a recent model proposing that these decisions are controlled by an opioid-based memory system residing in the NAc-S.

neuroscience↗

bioRxiv2024

Dorsomedial striatal neuroinflammation causes excessive goal-directed action control by disrupting homeostatic astrocyte function

Compulsive actions are typically thought to reflect the dominance of habits over goal-directed action. To address this, we mimicked the striatal neuroinflammation that is frequently exhibited in individuals with compulsive disorders in rats, by injecting the endotoxin lipopolysaccharide into the posterior dorsomedial striatum, and assessed the consequences for behavioural control. Surprisingly, this manipulation caused rats to acquire and maintain goal-directed actions under conditions that would otherwise produce habits. Immunohistochemical analyses indicated that these behaviours were a result of astrocytic proliferation. To probe this further, we chemogenetically activated the Gi-pathway in striatal astrocytes, which altered the firing properties of nearby medium spiny neurons and modulated goal-directed action control. Together, results show that striatal neuroinflammation is sufficient to bias action selection toward excessive goal-directed control via dysregulated astrocyte function. If translatable, our findings suggest that, contrary to conventional views, individuals with striatal neuroinflammation might be more prone to maladaptive goal-directed actions than habits, and future interventions should aim to restore appropriate action control.

animal behavior and cognition↗

bioRxiv2023

Immp2l knockdown increases stimulus-driven instrumental behaviour but does not alter goal-directed learning or neuron density in cortico-striatal circuits in a mouse model of Tourette syndrome and autism spectrum disorder

Cortico-striatal neurocircuits mediate goal-directed and habitual actions which are necessary for adaptive behaviour. It has recently been proposed that some of the core symptoms of autism spectrum disorder (ASD) and Gilles de la Tourette syndrome (GTS), such as tics and repetitive behaviours, may emerge because of imbalances in these neurocircuits. We have recently developed a model of ASD and GTS by knocking down Immp2l, a mitochondrial gene frequently associated with these disorders. The current study sought to determine whether Immp2l knockdown (KD) in male mice alters flexible, goal- or cue-driven behaviour using procedures specifically designed to examine response-outcome and stimulus-response associations, which underlie goal-directed and habitual behaviour, respectively. Whether Immp2l KD alters neuron density in cortico-striatal neurocircuits known to regulate these behaviours was also examined. Immp2l KD mice and wild type-like mice (WT) were trained on Pavlovian and instrumental learning procedures where auditory cues predicted food delivery and lever-press responses earned a food outcome. It was demonstrated that goal-directed learning was not changed for Immp2l KD mice compared to WT mice, as lever-press responses were sensitive to changes in the value of the food outcome, and to contingency reversal and degradation. There was also no difference in the capacity of KD mice to form habitual behaviours compared to WT mice following extending training of the instrumental action. However, Immp2l KD mice were more responsive to auditory stimuli paired with food as indicated by a non-specific increase in lever response rates during Pavlovian-to-instrumental transfer. Finally, there were no alterations to neuron density in striatum or any prefrontal cortex or limbic brain structures examined. Thus, the current study suggests that Immp2l is not necessary for learned maladaptive goal or stimulus driven behaviours in ASD or GTS, but that it may contribute to increased capacity for external stimuli to drive behaviour. Alterations to stimulus-driven behaviour could potentially influence the expression of tics and repetitive behaviours, suggesting that genetic alterations to Immp2l may contribute to these core symptoms in ASD and GTS. Given that this is the first application of this battery of instrumental learning procedures to a mouse model of ASD or GTS, it is an important initial step in determining the contribution of known risk-genes to goal-directed versus habitual behaviours, which should be more broadly applied to other rodent models of ASD and GTS in the future.

neuroscience↗

bioRxiv2022

CRF receptor type 1 modulates the nigrostriatal dopamine projection and facilitates cognitive flexibility after acute and chronic stress

Chronic unpredictable stress (CUS) impairs cognitive flexibility in rats, particularly when faced with additional mild acute stress (AS). We tested the hypothesis that this impairment is associated with alterations in dopamine activity in the dorsal striatum driven by corticotropin-releasing-factor receptor type 1 (CRFR1) in the substantia nigra pars compacta (SNpc). In experiment 1, rats received CUS or handling for 14 days, before learning two action-outcome associations (lever presses and food rewards). Learning was assessed using outcome devaluation. Cognitive flexibility was then assessed by reversing the outcome identities followed by a second outcome devaluation test, with half of the rats in each group receiving AS prior to reversal training. Dopamine and its metabolite were quantified in the dorsal striatum and CRFR1 mRNA was quantified in the SNpc. Increased dopaminergic activity in the left dorsal striatum and CRFR1 expression in the left SNpc were associated with resilience to AS in naive rats but with impairment in CUS+AS rats, suggesting a transition in hemispheric control from left to right as a protective mechanism following CUS. This suggestion was tested in experiment 2, where SNpc CRFR1 was blocked unilaterally prior to AS and reversal training. Blocking CRFR1 in the left medial SNpc impaired cognitive flexibility following AS in naive rats but restored it in CUS rats. Blocking CRFR1 in the left, but not right, lateral SNpc also impaired cognitive flexibility following AS in naive rats but had no effect in CUS rats.

neuroscience↗

bioRxiv2022

Cognitive effects of thalamostriatal degeneration are ameliorated by normalizing striatal cholinergic activity

The loss of neurons in parafascicular thalamus (Pf) and of their inputs to dorsomedial striatum (DMS) are associated with Lewy body disease (LBD) and Parkinsons disease dementia (PDD) and have been linked to the effects of neuroinflammation. In rats, these inputs regulate the function of striatal cholinergic interneurons (CINs) that are necessary for the flexible encoding of the action-outcome (AO) associations for goal-directed action. We found that these inputs modify the encoding, not retrieval, of new AO associations and cause burst-pause firing of CINs in the DMS during AO remapping. These adaptive effects were abolished by neuroinflammation in the Pf, resulting in the loss of goal-directed control when the rats were required to update AO associations after a change in contingency. We found that the neuronal and behavioral deficits induced by inflammation in the Pf were rescued by administration of selegiline, a MAO-B inhibitor that we found also enhances ATPase activity in CINs, suggesting a potential treatment for cognitive deficits associated with inflammation affecting the function of midline thalamic nuclei and related structures.

neuroscience↗

bioRxiv2022

Action sequence dynamics rely on accessory hyperdirect pathway-related targets in the striatum

Performing several actions in swift succession is often necessary to exploit known contingencies in the environment. However, in order to remain successful when contingency rules change, streamlined action sequences must be adaptable. Here, by combining analyses of behavioural microstructure with circuit-specific manipulation in mice, we report on a relationship between action timing variability and successful adaptation that relies on post-synaptic targets of primary motor cortical (M1) projections to dorsolateral striatum (DLS). Using a two-lever instrumental task, we found that mice build successful action sequences by first establishing action scaffolds, from which they dynamically elongate as task requirements extend. Specific interruption of the M1[->]DLS circuit altered these dynamics, prompting actions that were less variable in their timing, overall reducing opportunities for success. Our results reveal a role for M1[->]DLS circuitry in setting the exploration/exploitation balance that is required for adaptively guiding the timing and success of instrumental action. Based on evidence from transsynaptic tracing experiments, we propose that such function may involve additional downstream subcortical processing relating to collateralisation of descending motor pathways to multiple basal ganglia centres.

neuroscience↗

bioRxiv2020

Amygdala-cortical control of striatal plasticity drives the acquisition of goal-directed action.

The acquisition of goal-directed action requires the encoding of specific action-outcome associations involving plasticity in the posterior dorsomedial striatum (pDMS). We first investigated the relative involvement of the major inputs to the pDMS argued to be involved in this learning-related plasticity, from prelimbic prefrontal cortex (PL) and from the basolateral amygdala (BLA). Using ex vivo optogenetic stimulation of PL or BLA terminals in pDMS, we found that goal-directed learning potentiated the PL input to direct pathway spiny projection neurons (dSPNs) bilaterally but not to indirect pathway neurons (iSPNs). In contrast, learning-related plasticity was not observed in the direct BLA-pDMS pathway. Using toxicogenetics, we ablated BLA projections to either pDMS or PL and found that only the latter was necessary for goal-directed learning. Importantly, transient inactivation of the BLA during goal-directed learning prevented the PL-pDMS potentiation of dSPNs, establishing that the BLA input to the PL is necessary for the corticostriatal plasticity underlying goal-directed learning.

neuroscience↗

bioRxiv2020

Striatal direct and indirect pathway neurons differentially control the encoding and updating of goal-directed learning.

The posterior dorsomedial striatum (pDMS) is necessary for goal-directed action, however the role of the direct (dSPN) and indirect (iSPN) spiny projection neurons in the pDMS in such action remains unclear. In this series of experiments, we examined the role of pDMS SPNs in goal-directed action and found that, whereas dSPNs were critical for goal-directed learning and for energizing the learned response, iSPNs were involved in updating that learning to support response flexibility. Instrumental training elevated expression of the plasticity marker Zif268 in dSPNs only, and chemogenetic suppression of dSPN activity during training prevented goal-directed learning. Unilateral optogenetic inhibition of dSPNs induced an ipsilateral response bias in goal-directed action performance. In contrast, although initial goal-directed learning was unaffected by iSPN manipulations, optogenetic inhibition of iSPNs, but not dSPNs, impaired the updating of this learning and attenuated response flexibility after changes in the action-outcome contingency.

neuroscience↗

bioRxiv2019

Goal-directed action transiently depends on action space

Rats use spatiotemporal features of the environment to navigate to a goal, but whether representations of action space are necessary for non-navigational goal-directed actions is unknown. We addressed this question by assessing goal-directed action control across contexts and under hippocampal inactivation and found that such actions do indeed rely on a representation of action space but only immediately after initial acquisition.\n\nOne Sentence SummaryGoal-directed actions depend on a hippocampal representation of action space immediately after initial encoding but not after a delay.

animal behavior and cognition↗

bioRxiv2019

D1 and D2 systems converge in the striatum to update goal-directed learning

Extinction learning allows animals to withhold voluntary actions that are no longer related to reward and so provides a major source of behavioral control. Although such learning is thought to depend on dopamine signals in the striatum, the way the circuits mediating goal-directed control are reorganized during new learning remains unknown. Here, by mapping a dopamine-dependent transcriptional activation marker in large ensembles of striatal projection neurons (SPNs) expressing dopamine receptor type 1 (D1-SPNs) or 2 (D2-SPNs) in mice, we demonstrate an extensive and dynamic D2- to D1-SPN trans-modulation across the dorsal striatum that is necessary for updating previous goal-directed learning. Our findings suggest that D2-SPNs suppress the influence of outdated D1-SPN plasticity within functionally relevant striatal territories to reshape volitional action.

neuroscience↗

bioRxiv2018

Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models

Neuroscience studies of human decision-making abilities commonly involve sub-jects completing a decision-making task while BOLD signals are recorded using fMRI. Hypotheses are tested about which brain regions mediate the effect of past experience, such as rewards, on future actions. One standard approach to this is model-based fMRI data analysis, in which a model is fitted to the behavioral data, i.e., a subjects choices, and then the neural data are parsed to find brain regions whose BOLD signals are related to the models internal signals. However, the internal mechanics of such purely behavioral models are not constrained by the neural data, and therefore might miss or mischaracterize aspects of the brain. To address this limitation, we introduce a new method using recurrent neural network models that are flexible enough to be jointly fitted to the behavioral and neural data. We trained a model so that its internal states were suitably related to neural activity during the task, while at the same time its output predicted the next action a subject would execute. We then used the fitted model to create a novel visualization of the relationship between the activity in brain regions at different times following a reward and the choices the subject subsequently made. Finally, we validated our method using a previously published dataset. We found that the model was able to recover the underlying neural substrates that were discovered by explicit model engineering in the previous work, and also derived new results regarding the temporal pattern of brain activity.

neuroscience↗

bioRxiv2018

Models that learn how humans learn: the case of depression and bipolar disorders

Computational models of learning and decision-making processes in the brain play an important role in many domains. Such models typically have a constrained structure and make specific assumptions about the underlying human learning processes; these may make them underfit observed behaviours. Here we suggest an alternative method based on learning-to-learn approaches, using recurrent neural networks (RNNs) as a flexible family of models that have sufficient capacity to represent the complex learning and decision-making strategies used by humans. In this approach, an RNN is trained to predict the next action that a subject will take in a decision-making task, and in this way, learns to imitate the processes underlying subjects choices and their learning abilities. We demonstrate the benefits of this approach with a new dataset containing behaviour of uni-polar depression (n=34), bipolar (n=33) and control (n=34) participants in a two-armed bandit task. The results indicate that the new approach is better than baseline reinforcement-learning methods in terms of overall performance and its capacity to predict subjects choices. We show that the model can be interpreted using off-policy simulations, and thereby provide a novel clustering of subjects learning processes - something that often eludes traditional approaches to modelling and behavioural analysis.

bioinformatics↗

bioRxiv2017

Learning the structure of the world: The adaptive nature of state-space and action representations in multi-stage decision-making

State-space and action representations form the building blocks of decision-making processes in the brain; states map external cues to the current situation of the agent whereas actions provide the set of motor commands from which the agent can choose to achieve specific goals. Although these factors differ across environments, it is not currently known whether or how accurately state and action representations are acquired by the agent because previous experiments have typically provided this information a priori through instruction or pre-training. Here we show that, in the absence of such a priori knowledge, state and action representations adapt to reflect the structure of the world. We used a sequential decision-making task in rats in which they were required to pass through multiple states before reaching the goal, and for which the number of states and how they map onto external cues were not known a priori. We found that, early in training, animals selected actions as if the task was not sequential and outcomes were the immediate consequence of the most proximal action. During the course of training, however, rats recovered the true structure of the environment and made decisions based on the expanded state-space, reflecting the multiple stages of the task. We found a similar pattern with actions; early in training animals only considered the execution of single actions whereas, after training, they created useful action sequences that expanded the set of available actions. We conclude that the profile of choices shows a gradual shift from simple representations of actions and states to more complex structures compatible with the structure of the world.

animal behavior and cognition↗

bioRxiv2017

The Algorithmic Neuroanatomy Of Action-Outcome Learning

Although it is well known that animals can encode the consequences of their actions and can use this information to control action selection and evaluation, it is not known what learning rules control action-outcome (AO) learning. Here we trained participants to encode specific AO associations whilst undergoing functional imaging (fMRI) and used computational modelling to evaluate competing models. This analysis revealed that a Kalman filter, which learned the unique causal effect of each action, best characterized AO learning and found the medial prefrontal cortex differentiated the unique effect of actions from background effects. We subsequently extended these findings to show that mPFC participated in a circuit with parietal cortex and caudate nucleus to segregate distinct contributions to AO learning. The results extend our understanding of goal-directed learning and demonstrate that sensitivity to the causal relationship between actions and outcomes guides goal-directed learning rather than contiguous state-action relations.

neuroscience↗

bioRxiv2017

Optimal Response Vigor and Choice Under Non-stationary Outcome Values

Within a rational framework a decision-maker selects actions based on the reward-maximisation principle which stipulates they acquire outcomes with the highest values at the lowest cost. Action selection can be divided into two dimensions: selecting an action from several alternatives, and choosing its vigor, i.e., how fast the selected action should be executed. Both of these dimensions depend on the values of the outcomes, and these values are often affected as more outcomes are consumed, and so are the actions. Despite this, previous works have addressed the computational substrates of optimal actions only in the specific condition that the values of outcomes are constant, and it is still unknown what the optimal actions are when the values of outcomes are non-stationary. Here, based on an optimal control framework, we derive a computational model for optimal actions under non-stationary outcome values. The results imply that even when the values of outcomes are changing, the optimal response rate is constant rather than decreasing. This finding shows that, in contrast to previous theories, the commonly observed changes in the actions cannot be purely attributed to the changes in the outcome values. We then prove that this observation can be explained based on the uncertainty about temporal horizons; e.g., in the case of experimental protocols, the session duration. We further show that when multiple outcomes are available, the model explains probability matching as well as maximisation choice strategies. The model provides, therefore, a quantitative analysis of optimal actions and explicit predictions for future testing.

animal behavior and cognition↗