bioRxiv Science⌕ Search

Biology subjects

Dayan, P.

Publications and source records attributed to Dayan, P..

At least 19 recordsLinked to original sources

Dynamics of dopamine transients in human olfactory epithelium during reward learning

Dopamine signaling is crucial for motivation, movement, and learning, but has remained beyond the reach of measurement in healthy humans. Here, we measured sub-second dopamine in the olfactory epithelium of healthy humans during two reward learning tasks with distinct reward types: liquid reward during a conditioning task and monetary reward during an economic exchange game. In both tasks, as trials progress, peak dopamine transients shift from rewarding outcomes back to reward-predictive cues. In the economic game, this pattern retrains quickly when reward magnitude or reward timing is changed without warning. These findings match the time transfer characteristics and amplitude dependence of temporal difference prediction error signals. The results also establish a new, minimally invasive route for recording fast dopamine signaling in healthy humans: the nose.

neuroscience↗

Deciding when to decide: How recency, urgency, risk, and bias shape human sequential decision-making: A case study across the obsessive-compulsive spectrum

Deciding when to stop gathering information and commit to a choice is a fundamental challenge in decision-making under uncertainty. Normative characterizations such as Partially Observable Markov Decision Processes (POMDPs) prescribe mathematically optimal stopping rules; however, human evidence gathering systematically departs from optimality. Pathological departures -- such as the excessive indecisiveness characteristic of obsessive-compulsive disorder (OCD) -- offer an important opportunity to investigate the cognitive mechanisms involved in stopping. We extend a POMDP framework to incorporate key candidate suboptimalities: a biased prior belief, transient evidence exaggeration, progressive forgetting, boosted costs of error, temporal regulation (patience and urgency), and misperception of a deadline. We evaluate this model in a pre-existing dataset comprising 105 participants spanning healthy controls, generalised anxiety disorder, and the OCD spectrum performing an information gathering task with controlled, stochastic, deadlines. Model comparison reveals that human sequential choices are broadly governed by subjective risk penalties and time-dependent urgency, with a smaller and less certain contribution from an over-weighting of recent evidence, which a random-effects comparison does not support at the population level. Individuals differ in how that over-weighting is implemented: in one deadline condition, subjects divide almost evenly between models carrying a transient exaggeration of the newest sample, models carrying progressive forgetting of older evidence, and models carrying no recency mechanism at all. Crucially, while risk sensitivity and choice stochasticity act as shared mechanisms across conditions, mechanisms such as belief bias and patience are more variable. Finally, using OCD as a clinical case study, we demonstrate that simulating choices from the fitted exaggeration model reproduces model-agnostic regression signatures of clinical indecision, which the forgetting and no-recency accounts do not. These findings offer a generative foundation for dissecting clinical departures in information gathering across the obsessive-compulsive spectrum.

neuroscience↗

Coordinated behavioural and neural variability in hippocampal reward coding

An environment can be interpreted in multiple ways, with implications for behaviour and neural representations and for revision in these when circumstances change. Using two-photon calcium imaging in mouse CA1 during navigation under structured reward-location uncertainty, we found correlations between variability in anticipatory behaviour and the organization of place fields with respect to position and reward. Greater spatial anticipation was associated with enhanced reward-centred coding and increased generalisability of reward-aligned representations.

neuroscience↗

Quantifying the information about uncertainty in neural population codes

The activity of neural populations typically encodes more information about sensory or motor variables than can be captured by point estimates of the variables. We present and compare two approaches to quantifying this additional or ancillary information and its relationship to uncertainty: the mutual information between activity and estimation error, and the Fisher information loss, which can be interpreted in terms of curvature in information geometry. We show that deviations from Gaussianity of estimation errors, including the long tails frequently observed in human behavioural tasks, are an expected corollary of the presence of ancillary information. However, populations with similar distributions of estimation error can differ substantially in their ancillary information content depending on the noise characteristics. For a given population tuning and noise model, our results quantify an upper bound on the information about uncertainty that can be obtained from population activity alone: behaviour demonstrating knowledge in excess of this bound would indicate access to a separate source of information about uncertainty. Finally, we contrast the effects of external noise and decreasing internal signal strength on ancillary information and the Gaussianity of errors. Our work directly relates knowledge about uncertainty to non-Gaussianity in sensory estimates, and establishes a coherent theoretical foundation for investigating the basis of metacognition in neural population activity. Author summaryThe brain processes sensory evidence about the external world via inherently noisy neural activity. As a result, behavioural judgments - such as estimating the direction of a moving object - are fundamentally uncertain. While animals, including humans, routinely use uncertainty to guide decisions under risk, how neural populations represent this uncertainty remains unclear. In this work, we show how the same neural activity used to decode a sensory variable can also provide information about the estimates reliability. We introduce a mathematical framework to quantify this "ancillary information" directly from a neural populations encoding model. We demonstrate that ancillary information predicts non-Gaussianity in estimation errors and sets an upper bound on metacognitive sensitivity (how accurately subjective confidence tracks performance). Crucially, we show that neural populations with distinct noise characteristics can yield near-identical estimation errors while providing very different degrees of uncertainty information. This highlights the importance of evaluating ancillary information, not just error patterns, when comparing competing models of sensory coding.

neuroscience↗

Dopamine dynamics in human anterior cingulate cortex during Pavlovian-instrumental conflict

Dopamine is believed to modulate not only instrumental learning about the link between states, actions, and outcomes but also reflexive behaviours, such as a Pavlovian bias to approach in rewarding states and freeze in aversive ones. We studied these dual roles in the human brain, by combining intracranial dopamine recordings from the anterior cingulate cortex (ACC)-- a region implicated in behavioural and cognitive control -- with a motivational Go/NoGo task involving conflict between instrumental and Pavlovian action selection. We found evidence that dopamine in the ACC is involved in evaluating whether Pavlovian responding should guide behaviour. This computational motif was observed across multiple task events, including in response to rewards and punishments, and in analyses based on a reinforcement learning model. Our results indicate that dopamine supports learning at the more abstract level of behavioural policies in addition to the more concrete levels of states and actions.

neuroscience↗

Decomposing response inhibition: a POMDP model

Inhibitory control is a core cognitive function whose competence varies across the population, with impairments often observed in psychiatric conditions such as attention deficit hyperactivity disorder (ADHD). The Stop Signal Task (SST) is a widely used paradigm for assessing this ability. However, conventional formalizations of SST performance, such as the independent race model, rely on assumptions that are frequently violated in modern experimental designs. Furthermore, they typically fit only mean reaction times, overlooking crucial trial-by-trial dynamics. To address these limitations, we formalize the SST as a partially observable Markov decision process (POMDP). This framework characterizes inhibitory control with two components: noisy perceptual inference regarding stimuli and optimal control balanced against potential costs. To fit this model to the Adolescent Brain Cognitive Development (ABCD) study baseline cohort (N = 3,567), we introduce Transformer-encoded Simulation-Based Inference (TeSBI). This end-to-end architecture learns compact, sequence-aware embeddings from raw behavioral data. It enables efficient, amortized inference of individual-level posteriors. Extensive validation confirms it extracts reliable and identifiable parameters. We identify distinct latent computational attributes associated with scores on ADHD questionnaires. Controlling for sex, IQ, and medication status, children with higher ADHD scores exhibit subtle but robust shifts in computational attributes. They show a reduction in go cue directional precision, alongside a blunted sensitivity to stop error, go error and time costs. The learned embedding space reveals a continuous manifold in which children with higher ADHD scores are heterogeneously distributed, rather than forming distinct disorder clusters. This indicates that similar clinical characteristics can emerge from diverse combinations of computational mechanisms, supporting a dimensional perspective on neurodiversity. Our end-to-end framework can be extended to a broader range of cognitive tasks. It offers a scalable, theory-driven solution for analyzing large-scale behavioral data.

neuroscience↗

Neural replay is connected to latent cause inference and supports fast generalization

Dynamic environments require generalizing knowledge through two computational mechanisms: Inferring clusters of equivalent observations and propagating reward expectations within a cluster. While neural replay has been linked to both, past work has not studied these processes simultaneously, leaving their relative importance for replay unclear. We used fMRI and computational modeling to investigate latent cause inference and replay in 52 participants during a value-based task with shared rewards across bandits. Participants learned this structure and achieved one-shot generalization after reversals, consistent with a latent cause inference process. fMRI during predecision pauses revealed backwards replay of shared-reward bandits in the visual cortex and medial temporal lobe. Trial-wise fluctuations in visual replay strength and content were better explained by value updates than by structure learning. Conversely, abstract reward structure representations were localized specifically to the MTL. These results demonstrate that on-task replay serves reward propagation within learned latent structures to facilitate fast generalization.

neuroscience↗

VTA dopamine neuron activity produces spatially organized value representations

What is the neural architecture by which dopamine (DA) determines choice? Reinforcement learning (RL) has suggested an algorithmic chain: prediction errors based on reward train predicted values for stimuli and actions, and thereby determine choice. Elements of this chain have been identified, but have yet to be convincingly assembled. Here we used optogenetic stimulation of putative prediction error neurons at the top of this chain - i.e., DA neurons in the ventral tegmental area (VTA) - and recorded the activity of thousands of neurons in their target structure - the striatum - in mice performing a task that dissociates VTA DA signalling, stimulus value, action value, and choice. Conventional RL models captured some features of the data, including DA-driven choice reinforcement and value correlates, but failed on two key observations: i) action and state value correlates were relatively weak in the VTAs main target, the nucleus accumbens (NAc), and ii) VTA DA could not reinforce actions in the absence of a reward-predictive cue (CS+). Instead, we find that VTA DA produces stimulus value representations in the NAc, which in turn generates action value representations downstream (e.g. the CP). We formalize these observations in a Conditioned Reinforcer model, where VTA DA specifically acts on stimuli rather than actions, and these stimulus value representations are used to drive choice reinforcement.

neuroscience↗

Comparative Computational Modeling of Approach-Avoidance Biases in Suicidal Populations via Hierarchical Bayesian Inference

Pavlovian "approach or avoid" impulses are critical behavioral biases that, in excess, are linked to multiple psychiatric conditions. To investigate how such biases contribute to suicidal thoughts and behaviors, we analyzed data from two clinical populations completing an aversive Go/NoGo task. This task disentangles motor action (Go or NoGo) from outcome valence (escape from, or avoidance of, an aversive stimulus), enabling the isolation of Pavlovian biases from instrumental learning processes. We compared multiple computational models that had previously been proposed to explain Pavlovian tendencies, including reinforcement learning, active inference, and drift diffusion-based approaches. We employed a hierarchical Bayesian inference procedure that treats model identity as a random factor at the individual level, allowing an unbiased determination of which mechanisms most accurately captured participants behavior. Across both datasets, models featuring Pavlovian context biases plus a value-decay mechanism best accounted for performance. By contrast, policy-based Pavlovian models and more complex approaches, such as those integrating working memory or active inference, were supported by fewer study participants. These findings suggest that reflexive biases exert a persistent influence on decision-making, and that value decay plays a critical role in shaping behavior over time. Our results demonstrate the importance of systematically comparing and accounting for relevant cognitive processes to explain observed task behaviors. Understanding the factors contributing to task performance may help clarify how Pavlovian tendencies relate to psychopathology, including, in our case, elevated suicide risk. Finally, we illustrate how a complete hierarchical model selection framework can be applied to identify the most plausible mechanisms underlying Pavlovian biases, offering a robust approach for advancing our understanding of task behaviors and establishing clinical utility in future studies. Author summaryAutomatic "approach or avoid" reactions shape behavior, particularly in stressful or negative situations. In this study, we explored how these reflex-like tendencies might contribute to suicidal thoughts and behaviors. Two clinical groups completed a computerized task measuring responses to unpleasant sounds. Participants made either active responses (pressing a button to stop a sound) or passive responses (refraining from pressing to avoid starting a sound), allowing us to examine the interplay of automatic impulses and learning from past experiences. Our analysis showed that behavior was best explained by a model combining stable "approach or avoid" impulses with a forgetting process that reduced reliance on past experiences over time. More complex models involving strategies or memory-based control were less effective. These findings suggest that individuals with suicidal tendencies may rely on persistent reflex-like behaviors and over-index recent outcomes, compromising their ability to learn in uncertain environmental conditions. Understanding these cognitive processes provides insights into why some individuals feel trapped in harmful patterns of thought and behavior. Our work highlights how identifying shared traits in clinical populations using model-based methods can inform targeted mental health interventions and improve our understanding of cognitive functioning across disorders.

neuroscience↗

Octopamine and tyramine dynamics predict learning rate phenotypes during associative conditioning in honey bees

Biogenic amines are fundamental for physiological homeostasis and behavioral control in both vertebrates and invertebrates. Monoamine neurotransmitters released in target brain regions conjointly regulate adaptive learning and plasticity. However, our understanding of these multi-analyte mechanisms remains nascent, in part due to limitations in measurement technology. Here, during associative conditioning in honey bees, we concurrently tracked sub-second fluctuations in octopamine, tyramine, dopamine, and serotonin in the antennal lobe, where plasticity influences odorant representations. By repeatedly pairing an odorant with subsequent sucrose delivery, we observed individual differences in the conditioned response to odor, which occurred after a variable number of pairings (learners) or not at all (nonlearners). The distinction between learners and non-learners was reflected in neurotransmitter responses across experimental conditions. Remarkably, the speed of learning - the number of pairings prior to a proboscis extension reflex - could be predicted from monoamine opponent signaling (octopamine-tyramine), from both the first presentation of the odorant alone, prior to any pairing with sucrose, and from the first conditioned response to the odorant, coming after a number of sucrose pairings. These results suggest monoamine signaling phenotypes may relate directly to the now widely-reported socially-relevant genetic differences in honey bee learning.

neuroscience↗

Low dimensional latent structure underlying the choices of mice

An impressive wealth of cognitive neuroscience tasks involves combining perceptual information with an estimate of the latent state of the environment to make a decision. Such tasks have driven the development of theoretically-motivated cognitive models which offer compactly parameterised, and thereby insightful, accounts of the internal processes by which this might happen. However, the very large amounts of data that can now be collected present a challenge and an opportunity for this frame-work. The challenge is that these models can be shown to underfit the data systematically (Nassar & Frank, 2016; Palminteri, Wyart, & Koechlin, 2017; Wilson & Collins, 2019) - particularly in how they characterise the effect of the latent state estimate. But this opens up the opportunity to extend the framework and employ richer, more highly parameterised and flexible models, in a data-driven manner (Dezfouli, Griffiths, Ramos, Dayan, & Balleine, 2019). Unfortunately, it is tremendously difficult to interpret these models, precisely because of their flexibility. Here, we follow a recent approach (Eckstein, Summerfield, Daw, & Miller, 2024), in which components of compact models are progressively replaced with more flexible homologues, but, where possible, the resulting insights are mapped back into theoretically-transparent forms. We prove the effectiveness of our scheme by applying it to one of the largest available decision-making data sets for mice - more than 300,000 choices from 139 subjects studied by the International Brain Lab (The International Brain Laboratory et al., 2021). We found widely generalisable phenomena such as notable effects of continual fluctuations in task engagement and systematic differences between learning and forgetting of chosen and unchosen options in determining the subjective state estimate. Our results show that combining theory-driven and data-driven methods can reveal cognitive processes that would have been difficult to discover using either method on its own.

animal behavior and cognition↗

Entorhinal cortex signals dimensions of past experience that can be generalised in a novel environment

No two situations are identical. They can be similar in some aspects but different in others. This poses a key challenge when attempting to generalise our experience from one situation to another. How do we distinguish the aspects that transfer across situations from those that do not? One hypothesis is that the entorhinal cortex (EC) meets this challenge by forming factorised representations that allow for increased neural similarity between events that share generalisable features. We tested this hypothesis using functional magnetic resonance imaging. Female and male participants (n=40) were trained to report behavioural sequences based on an underlying graph structure. Participants then made decisions in a new environment where some, but not all graph transitions from the previous structure could be generalised. Behavioural results showed that participants distinguished the generalisable transition information. Accuracy was significantly higher in blocks where sequence transitions were shared across environments, than those in which transitions differed. This boost in accuracy was especially pronounced during early exposure to the novel environment. Throughout this early phase, neural patterns in EC showed a corresponding differentiation of the generalisable aspects. Neural patterns representing starting locations in familiar and novel environments were significantly more similar in EC on trials where sequences could be generalised from prior experience, compared to trials with new sequential transitions. This signalling was associated with improved performance when prior sequence knowledge could be reused. Our results suggest that during early exposure to novel environments, EC may signal dimensions of past experience that can be generalised. Significance StatementGeneralisation is the process of using our experience to solve new challenges. A central problem we face when attempting to generalise is determining which aspects of a new situation are similar to what we have experienced before and which are different. This allows us to generalise our knowledge selectively, transferring insights to the aspects that are similar and relevant in novel scenarios, while avoiding over-generalisation to the aspects that differ. The results from this experiment suggest that the entorhinal cortex may distinguish these aspects during early exposure to new environments, supporting complex generalisation behaviour.

neuroscience↗

Absence of Systematic Effects of Internalizing Psychopathology on Learning Under Uncertainty

Difficulties in adapting learning to meet the challenges of uncertain and changing environments are widely thought to play a central role in internalizing psychopathology, including anxiety and depression. This view stems from findings linking trait anxiety and transdiagnostic internalizing symptoms to learning impairments in laboratory tasks often used as proxies for real-world behavioral flexibility. These tasks typically require learners to adjust learning rates dynamically in response to uncertainty, for instance, increasing learning from prediction errors in volatile environments. However, prior studies have produced inconsistent and sometimes contradictory findings regarding the nature and extent of learning impairments in populations with internalizing disorders. To address this, we conducted eight experiments (N = 820) using predictive inference and reversal learning tasks, and applied a bi-factor analysis to capture internalizing symptom variance shared across and differentiated between anxiety and depression. While we observed robust evidence for adaptive learning-rate modulation across participants, we found no convincing evidence of a systematic relationship between internalizing symptoms and either learning rates or task performance. These findings challenge prominent claims that learning difficulties are a hallmark feature of internalizing psychopathology and suggest that the relationship between these traits and adaptive behavior under uncertainty may be more subtle than previously thought.

neuroscience↗

Homeostasis After Injury: How Intertwined Inference and Control Underpin Post-Injury Pain and Behaviour

Injuries are an unfortunate but inevitable fact of life, leading to an evolutionary mandate for powerful homeostatic processes of recovery and recuperation. The physiological responses of the body and the immune system must be coordinated with behaviour to allow protected time for this to happen, and to prevent further damage to the affected bodily parts. Reacting appropriately requires an internal control system that represents the nature and state of the injury and specifies and withholds actions accordingly. We bring the formal uncertainties embodied in this system into the framework of a partially observable Markov decision process. We discuss nociceptive phenomena in light of this analysis, noting particularly the counter-intuitive behaviours associated with injury investigation, and the propensity for transitions from normative, tonic, to pathological, chronic pain states. Importantly, these simulation results provide a quantitative account and enable us to sketch a much needed roadmap for future theoretical and experimental studies on injury, tonic pain, and the transition to chronic pain. AO_SCPLOWUTHORC_SCPLOW SO_SCPLOWUMMARYC_SCPLOWSeveral arguments suggest that the brain has a dedicated representation of the state of an injury. This provides an internal control system to modulate our behaviour, influencing the pain, anxiety, and mood changes appropriate to the need for heightened protection and recuperation during healing. Here, we propose a computational architecture for how this system might be constructed. We model the injury as a problem where its true state is only partially observable, requiring the brain to combine inference with optimal control to make decisions. We show how this framework offers an explanation of two core paradoxical observations: it explains behaviours such as rubbing or probing an injured area so as to gain information and reduce uncertainty, and it accounts for the high propensity of an injury to transition into a pathological, chronic pain state via information restriction. Overall, this provides a quantitative framework for mapping the bodys healing processes to their underlying neural substrates. This has the potential to help researchers identify novel targets for treating and possibly preventing chronic pain.

neuroscience↗

Composing egocentric and allocentric maps for flexible navigation

Egocentric representations of the environment have historically been relegated to being used only for simple forms of spatial behaviour such as stimulus-response learning. However, in the many cases that critical aspects of policies are best defined relative to the self, egocentric representations can be advantageous. Furthermore, there is evidence that forms of egocentric representation might exist in the wider hippocampal formation. Nevertheless, egocentric representations have yet to be fully incorporated as a component of modern navigational methods. Here we investigate egocentric successor representations (SRs) and their combination with allocentric representations. We build a reinforcement learning agent that combines an egocentric SR with a conventional allocentric SR to navigate complex 2D environments. We demonstrate that the agent learns generalisable egocentric and allocentric value functions which, even when only additively composed, allow it to learn policies efficiently and to adapt to new environments quickly. Our work shows the benefit for the hippocampal formation to capture egocentric, as well as allocentric, relational structure - and we link the egocentric SR to findings in the lateral entorhinal cortex. We offer a new perspective on how cognitive maps could usefully be composed from multiple simple maps representing associations between state features defined in different reference frames.

neuroscience↗

Neural signatures of model-based and model-free reinforcement learning across prefrontal cortex and striatum

Animals integrate knowledge about how the state of the environment evolves to choose actions that maximise reward. Such goal-directed behaviour - or model-based (MB) reinforcement learning (RL) - can flexibly adapt choice to changes, being thus distinct from simpler habitual - or model-free (MF) RL - strategies. Previous inactivation and neuroimaging work implicates prefrontal cortex (PFC) and the caudate striatal region in MB-RL; however, details are scarce about its implementation at the single-neuron level. Here, we recorded from two PFC regions - the dorsal anterior cingulate cortex (ACC) and dorsolateral PFC (DLPFC), and two striatal regions, caudate and putamen - while two rhesus macaques performed a sequential decision-making (two-step) task in which MB-RL involves knowledge about the statistics of reward and state transitions. All four regions, but particularly the ACC, encoded the rewards received and tracked the probabilistic state transitions that occurred. However, ACC (and to a lesser extent caudate) encoded the key variables of the task - namely the interaction between reward, transition and choice - which underlies MB decision-making. ACC and caudate neurons also encoded MB-derived estimates of choice values. Moreover, caudate value estimates of the choice options flipped when a rare transition occurred, demonstrating value update based on structural knowledge of the task. The striatal regions were unique (relative to PFC) in encoding the current and previous rewards with opposing polarities, reminiscent of dopaminergic neurons, and indicative of a MF prediction error. Our findings provide a deeper understanding of selective and temporally dissociable neural mechanisms underlying goal-directed behaviour.

neuroscience↗

Navigating uncertainty: reward location variability induces reorganization of hippocampal spatial representations

Navigating uncertainty is crucial for survival, with the location and availability of reward varying in different and unsignalled ways. Hippocampal place cell populations over-represent salient locations in an animals environment, including those associated with rewards; however, how the spatial uncertainties impact the cognitive map is unclear. We report a virtual spatial navigation task designed to test the impact of different levels and types of uncertainty about reward on place cell populations. When the reward location changed on a trial-by-trial basis, inducing expected uncertainty, a greater proportion of place cells followed along, and the reward and the track end became anchors of a warped spatial metric. When the reward location then unexpectedly moved, the fraction of reward place cells that followed was greater when starting from a state of expected, compared to low, uncertainty. Overall, we show that different forms of potentially interacting uncertainty generate remapping in parallel, task-relevant, reference frames.

neuroscience↗

A decision-theoretic model of multistability: perceptual switches as internal actions

Perceptual multistability has been studied for centuries using a diverse collection of approaches. Insights derived from this phenomenon range from core principles of information processing, such as perceptual inference, to high-level concerns, such as visual awareness. The dominant computational explanations of perceptual multistability are based on the Helmholtzian view of perception as inverse inference. However, these approaches struggle to account for the crucial role played by value, e.g., with percepts paired with reward dominating for longer periods than unpaired ones. In this study, we formulate perceptual multistability in terms of dynamic, value-based, choice, employing the formalism of a partially observable Markov decision process (POMDP). We use binocular rivalry as an example, considering different explicit and implicit sources of reward (and punishment) for each percept. The resulting values are time-dependent and influenced by novelty as a form of exploration. The solution of the POMDP is the optimal perceptual policy, and we show that this can replicate and explain several characteristics of binocular rivalry, ranging from classic hallmarks such as apparently spontaneous random switches with approximately gamma-distributed dominance periods to more subtle aspects such as the rich temporal dynamics of perceptual switching rates. Overall, our decision-theoretic perspective on perceptual multistability not only accounts for a wealth of unexplained data, but also opens up modern conceptions of internal reinforcement learning in service of understanding perceptual phenomena, and sensory processing more generally.

neuroscience↗