Biology subjects

Cohen, J. Y.

Publications and source records attributed to Cohen, J. Y..

13 recordsLinked to original sources

bioRxiv2026

Cell-type-specific sustained value representations in the claustrum

Flexible decision-making relies on interactions between frontal cortex and subcortical structures. The claustrum, a subcortical nucleus highly interconnected with frontal cortex, influences cortical activity and has been implicated in cognitive functions. Recording from claustrum neurons as mice performed a reinforcement learning task, we found that the activity of almost half of recorded neurons scaled with reward rate and predicted trial- by-trial adjustments in reaction time and choice switching. Individual neurons sustained this activity over seconds between trials. Our recordings identified two electrophysiologically distinct populations. One was excited during task execution and bidirectionally scaled its activity with reward rate. The other was suppressed during task execution, scaled activity inversely with reward rate and projected to frontal cortex, indicating that claustrocortical outputs produce graded increases in activity with decreasing reward rate. Our results identify the claustrum as a subcortical locus for stable value representations and integrate it into neuronal circuits for value-based decision-making.

neuroscience↗

bioRxiv2023

Serotonin predictively encodes value

The in vivo responses of dorsal raphe nucleus (DRN) serotonin neurons to emotionally-salient stimuli are a puzzle. Existing theories centred on reward, surprise, or uncertainty individually account for some aspects of serotonergic activity but not others. Here we find a unifying perspective in a biologically-constrained predictive code for cumulative future reward, a quantity called state value in reinforcement learning. Through simulations of trace conditioning experiments common in the serotonin literature, we show that our theory, called value prediction, intuitively explains phasic activation by both rewards and punishments, preference for surprising rewards but absence of a corresponding preference for punishments, and contextual modulation of tonic firing--observations that currently form the basis of many and varied serotonergic theories. Next, we re-analyzed data from a recent experiment and found serotonin neurons with activity patterns that are a surprisingly close match: our theory predicts the marginal effect of reward history on population activity with a precision <<0.1 Hz neuron-1. Finally, we directly compared against quantitative formulations of existing ideas and found that our theory best explains both within-trial activity dynamics and trial-to-trial modulations, offering performance usually several times better than the closest alternative. Overall, our results show that previous models are not wrong, but incomplete, and that reward, surprise, salience, and uncertainty are simply different faces of a predictively-encoded value signal. By unifying previous theories, our work represents an important step towards understanding the potentially heterogeneous computational roles of serotonin in learning, behaviour, and beyond.

neuroscience↗

bioRxiv2022

Two types of locus coeruleus norepinephrine neurons drive reinforcement learning

The cerebral cortex generates flexible behavior by learning. Reinforcement learning is thought to be driven by error signals in midbrain dopamine neurons. However, they project more densely to basal ganglia than cortex, leaving open the possibility of another source of learning signals for cortex. The locus coeruleus (LC) contains most of the brains norepinephrine (NE) neurons and project broadly to cortex. We measured activity from identified mouse LC-NE neurons during a behavioral task requiring ongoing learning from reward prediction errors (RPEs). We found two types of LC-NE neurons: neurons with wide action potentials (type I) were excited by positive RPE and showed an increasing relationship with change of choice likelihood. Neurons with thin action potentials (type II) were excited by lack of reward and showed a decreasing relationship with change of choice likelihood. Silencing LC-NE neurons changed future choices, as predicted from the electrophysiological recordings and a model of how RPEs are used to guide learning. We reveal functional heterogeneity of a neuromodulatory system in the brain and show that NE inputs to cortex act as a quantitative learning signal for flexible behavior.

neuroscience↗

bioRxiv2022

Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of entropy-based metrics in studying adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.

neuroscience↗

bioRxiv2021

Norepinephrine Potentiates and Serotonin Depresses Visual Cortical Responses by Transforming Eligibility Traces

Reinforcement allows organisms to learn which stimuli predict subsequent biological relevance. Hebbian mechanisms of synaptic plasticity are insufficient to account for reinforced learning because neuromodulators signaling biological relevance are delayed with respect to the neural activity associated with the stimulus. A theoretical solution is the concept of eligibility traces (eTraces), silent synaptic processes elicited by activity which upon arrival of a neuromodulator are converted into a lasting change in synaptic strength. Previously we demonstrated in visual cortical slices the Hebbian induction of eTraces and their conversion into LTP and LTD by the retroactive action of norepinephrine and serotonin Here we show in vivo in V1 that the induction of eTraces and their conversion to LTP/D by norepinephrine and serotonin respectively potentiates and depresses visual responses. We also show that the integrity of this process is crucial for ocular dominance plasticity, a canonical model of experience-dependent plasticity.

neuroscience↗

bioRxiv2021

Novel entropy-based metrics for predicting choice behavior based on local response to reward

For decades, behavioral scientists have used the matching law to quantify how animals distribute their choices between multiple options in response to reinforcement they receive. More recently, many reinforcement learning (RL) models have been developed to explain choice by integrating reward feedback over time. Despite reasonable success of RL models in capturing choice on a trial-by-trial basis, these models cannot capture variability in matching. To address this, we developed novel metrics based on information theory and applied them to choice data from dynamic learning tasks in mice and monkeys. We found that a single entropy-based metric can explain 50% and 41% of variance in matching in mice and monkeys, respectively. We then used limitations of existing RL models in capturing entropy-based metrics to construct a more accurate model of choice. Together, our novel entropy-based metrics provide a powerful, model-free tool to predict adaptive choice behavior and reveal underlying neural mechanisms.

neuroscience↗

bioRxiv2021

Reinforcement learning modeling reveals a reward-history-dependent strategy underlying reversal learning in squirrel monkeys

Insight into psychiatric disease and development of therapeutics relies on behavioral tasks that study similar cognitive constructs in multiple species. The reversal learning task is one popular paradigm that probes flexible behavior, aberrations of which are thought to be important in a number of disease states. Despite widespread use, there is a need for a high-throughput primate model that can bridge the genetic, anatomic, and behavioral gap between rodents and humans. Here, we trained squirrel monkeys, a promising preclinical model, on an image-guided deterministic reversal learning task. We found that squirrel monkeys exhibited two key hallmarks of behavior found in other species: integration of reward history over many trials and a side-specific bias. We adapted a reinforcement learning model and demonstrated that it could simulate monkey-like behavior, capture training-related trajectories, and provide insight into the strategies animals employed. These results validate squirrel monkeys as a model in which to study behavioral flexibility.

animal behavior and cognition↗

bioRxiv2021

Neuron-type specificity of dorsal raphe projections to ventral tegmental area

The midbrain dorsal raphe (DR) and ventral tegmental area (VTA) contain two of the brains main ascending neuromodulatory transmitters: serotonin and dopamine. We studied the pathway from DR to VTA using single-cell RNA sequencing, anatomical tracing, and electrophysiology and behavior in mice. Single-cell sequencing confirmed a differential distribution of dopamine cell types between medial and lateral aspects of the VTA. This molecular diversity included differential expression of a subset of glutamatergic and serotonergic receptors. Anatomical data showed that distinct serotonergic and glutamatergic populations of DR neurons project to distinct medial-lateral locations in VTA. Physiological data showed that serotonergic neurons are positioned to excite putative dopaminergic neurons in lateral VTA on short timescales (within trial), and inhibit them on long timescales (on the next trial). Our results reveal precise anatomical specificity of DR projections to VTA, and suggest a functional role for serotonergic modulation of dopaminergic function across multiple timescales.

neuroscience↗

bioRxiv2020

Serotonin neurons modulate learning rate through uncertainty

Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change is difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task, as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.

neuroscience↗

bioRxiv2020

Locus coeruleus spiking differently correlates with somatosensory cortex activity and pupil diameter

We examined the relationships between activity in the locus coeruleus (LC), activity in the primary somatosensory cortex (S1), and pupil diameter in mice performing a tactile detection task. While LC spiking consistently preceded S1 membrane potential depolarization and pupil dilation, the correlation between S1 and pupil was more heterogeneous. Furthermore, the relationships between LC, S1 and pupil varied on timescales of sub-seconds to seconds within trials. Our data suggest that pupil diameter can be dissociated from LC spiking and cannot be used as a stationary index of LC activity.

neuroscience↗

bioRxiv2020

Punishment history biases corticothalamic responses to motivationally-significant stimuli

Making predictions about future rewards or punishments is fundamental to adaptive behavior. These processes are influenced by prior experience. For example, prior exposure to aversive stimuli or stressors changes behavioral responses to negative- and positive-value predictive cues. Here, we demonstrate a role for medial prefrontal cortex (mPFC) neurons projecting to the paraventricular nucleus of the thalamus (PVT; mPFC[->]PVT) in this process. We found that a history of punishments negatively biased behavioral responses to motivationally-relevant stimuli in mice and that this negative bias was associated with hyperactivity in mPFC[->]PVT neurons during exposure to those cues. Furthermore, artificially mimicking this hyperactive response with selective optogenetic excitation of the same pathway recapitulated the punishmentinduced negative behavioral bias. Together, our results highlight how information flow within the mPFC[->]PVT circuit is critical for making predictions about imminent motivationally-relevant outcomes as a function of prior experience.

neuroscience↗

bioRxiv2020

A Clock-Driven Neural Network Critical for Arousal

The daily cycling of sleep and arousal states is among the most prominent biological rhythms under circadian control. While much is known about the core circadian clock1,2, how this clock tunes sleep and arousal remains poorly understood3. In Drosophila, we previously characterized WIDE AWAKE (WAKE), a clock-output molecule that promotes sleep at night4,5. Here, we show that the function of WAKE in regulating circadian-dependent neural excitability and arousal is conserved in mice. mWake+ cells are found in the suprachiasmatic nucleus (SCN) and dorsomedial hypothalamus (DMH). mWakeDMH neurons drive wakefulness and exhibit rhythmic spiking, with greater firing during the night vs the day. Loss of mWAKE leads to increased spiking of mWake+ SCN and DMH neurons and prominent behavioural arousal, specifically during the night. Single-cell sequencing, imaging, and patch-clamp experiments reveal that mWakeDMH neurons constitute a glutamatergic/GABAergic population that projects widely, receives neuromodulatory input, and acts on neuromodulatory neurons. Strikingly, broad chemogenetic silencing of mWake+ cells leads to profound loss of behavioural responsiveness and low amplitude, low frequency electroencephalography waveforms. These findings suggest that the genetic mechanisms regulating circadian control of sleep and arousal are conserved across >500 million years of evolution and define a clock-regulated neural network critical for arousal.

neuroscience↗

bioRxiv2019

A history-derived reward prediction error signal in ventral pallidum

Learning from past interactions with the environment is critical for adaptive behavior. Within the framework of reinforcement learning, the nervous system builds expectations about future reward by computing reward prediction errors (RPEs), the difference between actual and predicted rewards. Correlates of RPEs have been observed in the midbrain dopamine system, which is thought to locally compute this important variable in service of learning. However, the extent to which RPE signals may be computed upstream of the dopamine system is largely unknown. Here, we quantify history-based RPE signals in the ventral pallidum (VP), an input region to the midbrain dopamine system implicated in reward-seeking behavior. We trained rats to associate cues with future delivery of reward and fit computational models to predict individual neuron firing rates at the time of reward delivery. We found that a subset of VP neurons encoded RPEs and did so more robustly than nucleus accumbens, an input to VP. VP RPEs predicted trial-by-trial task engagement, and optogenetic inhibition of VP reduced subsequent task-related reward seeking. Consistent with reinforcement learning, activity of VP RPE cells adapted when rewards were delivered in blocks. We further found that history- and cue-based RPEs were largely separate across the VP neural population. The presence of behaviorally-instructive RPE signals in the VP suggests a pivotal role for this region in value-based computations.

neuroscience↗