Biology subjects

Constantinople, C. M.

Publications and source records attributed to Constantinople, C. M..

11 recordsLinked to original sources

bioRxiv2026

A corticostriatal circuit updates subjective beliefs about latent task states

Beliefs about states of the world profoundly impact decision-making and learning, but little is known about how neural circuits represent and update beliefs. We performed projection-specific recordings and perturbations from neurons in the orbitofrontal cortex (OFC) projecting to the intermediate or rostral cau-date putamen (CPi/CPr) in rats performing a task with hidden reward states. Stimulating OFC[->]CPi neurons biased rats beliefs towards high reward states. Recordings from optogenetically-tagged OFC[->]CPi neurons showed that they encoded categorical evidence for high reward states, shaped by a saturating non-linearity in neural responses. Downstream neurons could, in principle, decode the full belief distribution over reward states as rats deliberated about deci-sions. Finally, projection-specific perturbations disrupted encoding of hidden states within OFC. These findings reveal the circuit implementation of a core cognitive computation, updating subjective beliefs about abstract latent states of the environment.

neuroscience↗

bioRxiv2024

Neural dynamics in the orbitofrontal cortex reveal cognitive strategies

While the orbitofrontal cortex (OFC) is implicated in learning and inferring latent states, the precise computation performed by OFC for state inference is unclear. Here we show that rat OFC updates beliefs about states, and this process is decipherable from OFC dynamics in rats performing state inference, but not alternative strategies. We trained rats to perform a temporal wagering task with hidden reward states. Well-trained rats used state inference when deciding how long to wait for rewards, and OFC inactivations impaired belief updating about states. Electrophysiology and novel population analysis methods identified latent neural factors reflecting inferred states in rats performing inference, but not other strategies. Neural firing rates and latent population factors showed abrupt changes following trials that were informative of state transitions. These results identify a precise computation performed by OFC, and reveal neural signatures of inference.

neuroscience↗

bioRxiv2024

Acetylcholine demixes heterogeneous dopamine signals for learning and moving

Midbrain dopamine neurons promote both reinforcement learning and movement vigour1-12. A major outstanding question is how dopamine-recipient neurons in the striatum parse these heterogeneous signals. Previous work suggests that cholinergic striatal interneurons may play a role, perhaps by gating dopamine-dependent plasticity13-16, but this has not been tested in behaving animals. Here we studied rats performing a decision-making task with reward- and movement-related events at distinct time points. Optical measurement of dopamine and acetylcholine release in the dorsomedial striatum (DMS) revealed distinct dynamics at task events. Reward cues evoked cholinergic pauses with different phase relationships relative to dopamine. When dopamine slightly lagged cholinergic dips, dopamine predicted future behaviour, and DMS firing rates on subsequent trials. In contrast, when dopamine slightly preceded cholinergic dips, there was no observable relationship between dopamine and learning. Finally, when dopamine was coincident with cholinergic bursts, it preceded and predicted the vigour of upcoming contralateral orienting movements. Our findings suggest that the precise phase relationship between dopamine and acetylcholine allows dopamine to be used for either movement or learning depending on instantaneous behavioural context.

neuroscience↗

bioRxiv2024

Mesolimbic dopamine encodes reward prediction errors independent of learning rates

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented in synaptic weights in the striatum, and updated by dopamine-dependent plasticity, suggesting that dopamine release might reflect the product of the learning rate and RPE. Here, we leveraged the fact that animals learn faster in volatile environments to characterize dopamine encoding of learning rates in the nucleus accumbens core (NAcc). We trained rats on a task with semi-observable states offering different rewards, and rats adjusted how quickly they initiated trials across states using RPEs. Computational modeling and behavioral analyses showed that learning rates were higher following state transitions, and scaled with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the NAcc encoded RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

neuroscience↗

bioRxiv2024

Curriculum learning inspired by behavioral shaping trains neural networks to adopt animal-like decision making strategies

1Recurrent neural networks (RNN) are ubiquitously used in neuroscience to capture both neural dynamics and behaviors of living systems. However, when it comes to complex cognitive tasks, training RNNs with traditional methods can prove difficult and fall short of capturing crucial aspects of animal behavior. Here we propose a principled approach for identifying and incorporating compositional tasks as part of RNN training. Taking as target a temporal wagering task previously studied in rats, we design a pretraining curriculum of simpler cognitive tasks that reflect relevant sub-computations. We show that this pretraining substantially improves learning efficacy and is critical for RNNs to adopt similar strategies as rats, including long-timescale inference of latent states, which conventional pretraining approaches fail to capture. Mechanistically, our pretraining supports the development of slow dynamical systems features needed for implementing both inference and value-based decision making. Overall, our approach is an important step for endowing RNNs with relevant inductive biases, which is important when modeling complex behaviors that rely on multiple cognitive computations.

neuroscience↗

bioRxiv2023

Estrogenic control of reward prediction errors and reinforcement learning

Gonadal hormones act throughout the brain1, and neuropsychiatric disorders vary in symptom severity over the reproductive cycle, pregnancy, and perimenopause2-4. Yet how hormones influence cognitive processes is unclear. Exogenous 17{beta}-estradiol modulates dopamine signaling in the nucleus accumbens core (NAcc)5,6, which instantiates reward prediction errors (RPEs) for reinforcement learning7-16. Here we show that endogenous 17{beta}-estradiol enhances RPEs and sensitivity to previous rewards by reducing dopamine reuptake proteins in the NAcc. Rats performed a task with different reward states; they adjusted how quickly they initiated trials across states, balancing effort against expected rewards. NAcc dopamine reflected RPEs that predicted and causally influenced initiation times. Elevated endogenous 17{beta}-estradiol increased sensitivity to reward states by enhancing dopaminergic RPEs in the NAcc. Proteomics revealed reduced dopamine transporter expression. Finally, knockdown of midbrain estrogen receptors suppressed reinforcement learning. 17{beta}-estradiol therefore controls RPEs via dopamine reuptake, mechanistically revealing how hormones influence neural dynamics for motivation and learning.

neuroscience↗

bioRxiv2023

Rats rapidly switch between retrospective and inferential value computations

The value of the environment determines animals motivational states and sets expectations for error-based learning1-3. How are values computed? Rein-forcement learning systems can store or "cache" values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single tri-als. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for re-wards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environ-ment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behav-iors.

neuroscience↗

bioRxiv2022

Transformation of acoustic information to sensory decision variables in the parietal cortex

The process by which sensory evidence contributes to perceptual choices requires an understanding of its transformation into decision variables. Here, we address this issue by evaluating the neural representation of acoustic information in auditory cortex-recipient parietal cortex while gerbils either performed an auditory discrimination task or while they passively listened to identical acoustic stimuli. During task performance, decoding performance of simultaneously recorded parietal neurons reflected psychometric sensitivity. In contrast, decoding performance during passive listening was significantly reduced. Principal component and geometric analyses each revealed the emergence of decision-relevant, linearly separable manifolds, but only during task engagement. Finally, using a clustering analysis, we found subpopulations of neurons that may reflect the encoding of separate segments during task performance: stimulus integration and motor preparation or execution. Taken together, our findings demonstrate how parietal cortex neurons integrate and transform encoded auditory information to guide sound-driven perceptual decisions.

neuroscience↗

bioRxiv2021

Subpopulations of neurons in lOFC encode previous and current rewards at time of choice

1Studies of neural dynamics in lateral orbitofrontal cortex (lOFC) have shown that subsets of neurons that encode distinct aspects of behavior, such as value, may project to common downstream targets. However, it is unclear whether reward history, which may subserve lOFCs well-documented role in learning, is represented by functional subpopulations in lOFC. Previously, we analyzed neural recordings from rats performing a value-based decision-making task, and we documented trial-by-trial learning that required lOFC (Constantinople et al., 2019). Here we characterize functional subpopulations of lOFC neurons during behavior, including their encoding of task variables. We found five distinct clusters of lOFC neurons, either based on clustering of their trial-averaged peristimulus time histograms (PSTHs), or a feature space defined by their average conditional firing rates aligned to different task variables. We observed weak encoding of reward attributes, but stronger encoding of reward history, the animals left or right choice, and reward receipt across all clusters. Only one cluster, however, encoded the animals reward history at the time shortly preceding the choice, suggesting a possible role in integrating previous and current trial outcomes at the time of choice. This cluster also exhibits qualitatively similar responses to identified corticostriatal projection neurons in a recent study (Hirokawa et al., 2019), and suggests a possible role for subpopulations of lOFC neurons in mediating trial-by-trial learning.

neuroscience↗

bioRxiv2019

Orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases

Individual choices are not made in isolation but are embedded in a series of past experiences, decisions, and outcomes. The effects of past experiences on choices, often called sequential biases, are ubiquitous in perceptual and value-based decision-making, but their neural substrates are unclear. We trained rats to choose between explicitly cued guaranteed and probabilistic rewards in a task in which outcomes on each trial were independent. Behavioral variability often reflected the effects of previous trials, including increased willingness to take risks following risky wins, and spatial \"win-stay/lose-shift\" biases. Electrophysiological recordings from lateral orbitofrontal cortex (lOFC) revealed strong encoding of reward history and receipt, and optogenetic inhibition of lOFC during choice reporting eliminated rats increased preference for risk on subsequent trials following risky wins, but spared other sequential effects. Our data show that different sequential biases are neurally dissociable, and that the lOFCs role in adaptive behavior promotes learning of more abstract, task-specific biases (here, biases for the risky option), but not spatial ones. These data are consistent with proposals that the OFC represents the animals location in an abstract representation of task states, and highlight the lOFCs role in learning.

neuroscience↗

bioRxiv2018

An Analysis of Decision Under Risk in Rats

Prospect Theory is the predominant behavioral economic theory describing decision-making under risk. It accounts for near universal aspects of human choice behavior whose prevalence may reflect fundamental neural mechanisms. We now apply Prospect Theorys framework to rodents, using a task in which rats chose between guaranteed and probabilistic rewards. Like humans, rats distorted probabilities and showed diminishing marginal sensitivity, in which they were less sensitive to differences in larger rewards. They exhibited reference dependence, in which the valence of outcomes (gain or loss) was determined by an internal reference point reflecting reward history. The similarities between rats and humans suggest conserved neural substrates, and enable application of powerful molecular/circuit tools to study mechanisms of psychological phenomena from behavioral economics.

neuroscience↗