Biology subjects

Dayan, P.

Publications and source records attributed to Dayan, P..

At least 37 records · Page 2Linked to original sources

bioRxiv2024

Distinct decision processes for 3D and motion stimuli in both humans and monkeys revealed by computational modelling

Decision-making models distil principles of information processing that underlie a range of cognitive functions. For the cases of motion detection in random-dot kinematograms and decisions about the rotation of 3D structure-from-motion cylinders, contributing neural processes have been localized to specific circuits in extrastriate area V5/MT on the basis of both causal and correlative evidence, suggesting a common decision path. Here, we arranged for humans and rhesus monkeys to make perceptual choices about these two stimulus types, indicating their decision with either hand or eye movements. In both species, the parameter distributions of a hierarchical Drift Diffusion Model (DDM) revealed systematic differences for stimulus type but not the different modes of response. States in a hidden Markov model also indicated distinct decision strategies, again for the two stimulus types but not for response mode. Although physiological evidence points to area V5/MT as a vehicle for relevant perceptual signals for both stimulus types, computational modelling of the present results reveal distinct decision processes, therefore predicting that different neural processes underlie judgements about motion and 3D-depth, consistently for humans and monkeys.

neuroscience↗

bioRxiv2024

A computational approach to understanding effort-based decision-making in depression

BackgroundMotivational dysfunction is a core feature of depression, and can have debilitating effects on everyday function. However, it is unclear which disrupted cognitive processes underlie impaired motivation, and whether impairments persist following remission. Decision-making concerning exerting effort to obtain rewards offers a promising framework for understanding motivation, especially when examined with computational tools which can offer precise quantification of latent processes. MethodsEffort-based decision-making was assessed using the Apple Gathering Task, where participants decide whether to exert effort via a grip-force device to obtain varying levels of reward; effort levels were individually calibrated and varied parametrically. We present a comprehensive computational analysis of decision-making, initially validating our model in healthy volunteers (N=67), before applying it in a case-control study including current (N=41) and remitted (N=46) unmedicated depressed individuals, and healthy volunteers with (N=36) and without (N=57) a family history of depression. ResultsFour fundamental computational mechanisms that drive patterns of effort-based decisions, which replicated across samples, were identified: overall bias to accept effort challenges; reward sensitivity; and linear and quadratic effort sensitivity. Traditional model-agnostic analyses showed that both depressed groups showed lower willingness to exert effort. In contrast with previous findings, computational analysis revealed that this difference was primarily driven by lower effort acceptance bias, but not altered effort or reward sensitivity. ConclusionsThis work provides insight into the computational mechanisms underlying motivational dysfunction in depression. Lower willingness to exert effort could represent a trait-like factor contributing to symptoms, and might represent a fruitful target for treatment and prevention.

neuroscience↗

bioRxiv2024

Multifaceted confidence in exploratory choice

Our choices are typically accompanied by a feeling of confidence - an internal estimate that they are correct. Correctness, however, depends on our goals. For example, exploration-exploitation problems entail a tension between short- and long-term goals: finding out about the value of one option could mean foregoing another option that is apparently more rewarding. Here, we hypothesised that after making an exploratory choice that involves sacrificing an immediate gain, subjects will be confident that they chose a better option for long-term rewards, but not confident that it was a better option for immediate reward. We asked 250 subjects across 2 experiments to perform a varying-horizon two-arm bandits task, in which we asked them to rate their confidence that their choice would lead to more immediate, or more total reward. Confirming previous studies, we found a significant increase in exploration with increasing trial horizon, but, contrary to our predictions, we found no difference between confidence in immediate or total reward. This dissociation is further evidence for a separation in the mechanisms involved in choices and confidence judgements.

neuroscience↗

bioRxiv2024

The choice-wide behavioral association study: data-driven identification of interpretable behavioral components

Behavior contains rich structure across many timescales. However, there is a dearth of methods to identify relevant components, especially over the longer periods required for learning and decision-making. Inspired by the goals and techniques of genome-wide association studies, we present a data-driven method--the choice-wide behavioral association study: CBAS--that systematically identifies such behavioral features. CBAS extracts sequences of actions or choices that either significantly differ between groups or correlate with a covariate of interest, using a powerful resampling-based method to correct for multiple comparisons. We illustrate CBAS through application to tasks performed by flies, rats and humans, showing that it provides unexpected and interpretable information in each case.

neuroscience↗

bioRxiv2024

Risking your Tail: Modeling Individual Differences in Risk-sensitive Exploration using Conditional Value at Risk and Bayes Adaptive Markov Decision Processes

Novelty is a double-edged sword for agents and animals alike: they might benefit from untapped resources or face unexpected costs or dangers such as predation. The conventional exploration/exploitation tradeoff is thus coloured by risk-sensitivity. A wealth of experiments has shown how animals solve this dilemma, for example using intermittent approach. However, there are large individual differences in the nature of approach, and modeling has yet to elucidate how this might be based on animals differing prior expectations about reward and threat, and differing degrees of risk aversion. To capture these factors, we built a Bayes adaptive Markov decision process model with three key components: an adaptive hazard function capturing potential predation, an intrinsic reward function providing the urge to explore, and a conditional value at risk (CVaR) objective, which is a contemporary measure of trait risk-sensitivity. We fit this model to a coarse-grain abstraction of the behaviour of 26 animals who freely explored a novel object in an open-field arena (Akiti et al. Neuron 110, 2022). We show that the model captures both quantitative (frequency, duration of exploratory bouts) and qualitative (with distinguished, cautious, tail-behind approach) features of behavior, including the substantial idiosyncrasies that were observed. Some animals begin with cautious exploration, and quickly transition to confident approach to maximize exploration for reward; we classify them as potentially more risk neutral, and enjoying a flexible hazard prior. By contrast, other animals only ever approach in a cautious manner and display a form of self-censoring; they are characterized by potential risk aversion and high and inflexible hazard priors. Explaining risk-sensitive exploration using factorized parameters of reinforcement learning models could aid in the understanding, diagnosis, and treatment of psychiatric abnormalities such as anxiety disorders.

animal behavior and cognition↗

bioRxiv2023

Dissecting the Complexities of Learning With Infinite Hidden Markov Models

Learning to exploit the contingencies of a complex experiment is not an easy task for animals. Individuals learn in an idiosyncratic manner, revising their approaches multiple times as they are shaped, or shape themselves, and potentially end up with different strategies. Their long-run learning curves are therefore a tantalizing target for the sort of individualized quantitative characterizations that sophisticated modelling can provide. However, any such model requires a flexible and extensible structure which can capture radically new behaviours as well as slow changes in existing ones. To this end, we suggest a dynamic input-output infinite hidden semi-Markov model, whose latent states are associated with specific components of behaviour. This model includes an infinite number of potential states and so has the capacity to describe substantially new behaviours by unearthing extra states; while dynamics in the model allow it to capture more modest adaptations to existing behaviours. We individually fit the model to data collected from more than 100 mice as they learned a contrast detection task over tens of sessions and around fifteen thousand trials each. Despite large individual differences, we found that most animals progressed through three major stages of learning, the transitions between which were marked by distinct additions to task understanding. We furthermore showed that marked changes in behaviour are much more likely to occur at the very beginning of sessions, i.e. after a period of rest, and that response biases in earlier stages are not predictive of biases later on in this task.

animal behavior and cognition↗

bioRxiv2023

Wagers for work: Decomposing the costs of cognitive effort

Some aspects of cognition are more taxing than others. Accordingly, many people will avoid cognitively demanding tasks in favor of simpler alternatives. Which components of these tasks are costly, and how much, remains unknown. Here, we use a novel task design in which subjects request wages for completing cognitive tasks and a computational modeling procedure that decomposes their wages into the costs driving them. Using working memory as a test case, our approach revealed that gating new information into memory and protecting against interference are costly. Critically, other factors, like memory load, appeared less costly. Other key factors which may drive effort costs, such as error avoidance, had minimal influence on wage requests. Our approach is sensitive to individual differences, and could be used in psychiatric populations to understand the true underlying nature of apparent cognitive deficits. Author SummaryAnyone who has tried to mentally calculate how much to tip at a restaurant knows that cognitive effort can feel aversive. Doing math in your head, like most high-level cognitive abilities, depends critically on working memory (WM). We know that WM is sometimes effortful to use, but we dont know which aspects of WM use drive these effort costs. To address this question, we had participants request wages in exchange for performing various tasks that differed in their specific WM demands. Using computational models of their wage demands, we demonstrated that some aspects of WM are costly, such as bringing new information into memory and preventing interference. Other factors, like the amount of information in memory and attempts to avoid mistakes, were less costly. Our approach identified which specific subcomponents of WM are aversive. Future research could use these methods to test theories about how motivational problems might be masquerading as cognitive deficits in psychiatric populations.

neuroscience↗

bioRxiv2023

A Brain-Wide Map of Neural Activity during Complex Behaviour

A key challenge in neuroscience is understanding how neurons in hundreds of interconnected brain regions integrate sensory inputs with prior expectations to initiate movements and thereby make decisions. It is difficult to meet this challenge if different laboratories apply different analyses to different recordings in different regions during different behaviours. Here, we report a comprehensive set of recordings from 621733 neurons across 139 mice in 12 labs performing a decision-making task with sensory, motor, and cognitive components, obtained with 699 Neuropixels probe insertions covering 279 brain areas in the left forebrain and midbrain and the right hindbrain and cerebellum. We provide an initial appraisal of this brain-wide map, assessing how neural activity encoded key task variables. Representations of visual stimuli appeared transiently in classical visual areas after stimulus onset and then spread to ramp-like activity in a collection of mid- and hindbrain regions that also encoded choices. Neural responses correlated with motor action almost everywhere in the brain. Responses to reward delivery and consumption versus negative feedback were also widespread. This publicly available dataset represents an unprecedented resource for understanding how computations distributed across and within brain areas drive behaviour.

neuroscience↗

bioRxiv2023

Reframing dopamine: Controlling the controller at the limbic-motor interface

Pavlovian influences notoriously interfere with operant behaviour; various such may be associated with the release of the neuromodulator dopamine in the nucleus accumbens. One role for cognitive control is suppressing these influences. Here, using the examples of active avoidance and omission behaviour, we examine the possibility that one instrument of control is direct manipulation of the dopamine signal itself.

neuroscience↗

bioRxiv2023

Exploring Replay

Exploration is vital for animals and artificial agents who face uncertainty about their environments due to initial ignorance or subsequent changes. Their choices need to balance exploitation of the knowledge already acquired, with exploration to resolve uncertainty [1, 2]. However, the exact algorithmic structure of exploratory choices in the brain still remains largely elusive. A venerable idea in reinforcement learning is that agents can plan appropriate exploratory choices offline, during the equivalent of quiet wakefulness or sleep. Although offline processing in humans and other animals, in the form of hippocampal replay and preplay, has recently been the subject of highly successful modelling [3-5], existing methods only apply to known environments. Thus, they cannot predict exploratory replay choices during learning and/or behaviour in dynamic environments. Here, we extend the theory of Mattar & Daw [3] to examine the potential role of replay in approximately optimal exploration, deriving testable predictions for the patterns of exploratory replay choices in a paradigmatic spatial navigation task. Our modelling provides a normative interpretation of the available experimental data suggestive of exploratory replay. Furthermore, we highlight the importance of sequence replay, and license a range of new experimental paradigms that should further our understanding of offline processing.

neuroscience↗

bioRxiv2022

Vigilance, arousal, and acetylcholine: Optimal control of attention in a simple detection task

Paying attention to particular aspects of the world or being more vigilant in general can be interpreted as forms of internal action. Such arousal-related choices come with the benefit of increasing the quality and situational appropriateness of information acquisition and processing, but incur potentially expensive energetic and opportunity costs. One implementational route for these choices is widespread ascending neuromodulation, including by acetylcholine (ACh). The key computational question that elective attention poses for sensory processing is when it is worthwhile paying these costs, and this includes consideration of whether sufficient information has yet been collected to justify the higher signal-to-noise ratio afforded by greater attention and, particularly if a change in attentional state is more expensive than its maintenance, when states of heightened attention ought to persist. We offer a partially observable Markov decision-process treatment of optional attention in a detection task, and use it to provide a qualitative model of the results of studies using modern techniques to measure and manipulate ACh in rodents performing a similar task.

neuroscience↗

bioRxiv2022

Tracking human skill learning with a hierarchical Bayesian sequence model

Humans can implicitly learn complex perceptuo-motor skills over the course of large numbers of trials. This likely depends on our becoming better able to take advantage of ever richer and temporally deeper predictive relationships in the environment. Here, we offer a novel characterization of this process, fitting a non-parametric, hierarchical Bayesian sequence model to the reaction times of human participants responses over ten sessions, each comprising thousands of trials, in a serial reaction time task involving higher-order dependencies. The model, adapted from the domain of language, forgetfully updates trial-by-trial, and seamlessly combines predictive information from shorter and longer windows onto past events, weighing the windows proportionally to their predictive power. As the model defines a prior over window depths, we were able to determine the extent to which the internal predictions of individual participant depended on how many previous elements. Already in the first session, the model showed that participants had begun to rely on two previous elements (i.e., trigrams), thereby successfully adapting to the most prominent higher-order structure in the task. The extent to which local statistical fluctuations influenced participants responses waned over subsequent sessions, as subjects forgot the trigrams less and evidenced skilled performance. By the eighth session, a subset of participants shifted their prior further to consider a context deeper than two previous elements. Finally, participants showed resistance to interference and slow forgetting of the old sequence when it was changed in the final sessions. Model parameters for individual subjects covaried appropriately with independent measures of working memory. In sum, the model offers the first principled account of the adaptive complexity and nuanced dynamics of humans internal sequence representations during long-term implicit skill learning. Author summaryA central function of the brain is to predict. One challenge of prediction is that both external events and our own actions can depend on a variably deep temporal context of previous events or actions. For instance, in a short motor routine, like opening a door, our actions only depend on a few previous ones (e.g., push the handle if the key was turned). In longer routines such as coffee making, our actions require a deeper context (e.g., place the moka pot on the hob if coffee is ground, the pot is filled and closed, and the hob is on). We adopted a model from the natural language processing literature that matches humans ability to learn variable-length relationships in sequences. This model explained the gradual emergence of more complex sequence knowledge and individual differences in an experiment where humans practiced a perceptual-motor sequence over 10 weekly sessions.

animal behavior and cognition↗

bioRxiv2022

The role of the orbitofrontal cortex in creating cognitive maps

We use internal models of the external world to guide behavior, but little is known about how these cognitive maps are created. The orbitofrontal cortex (OFC) is typically thought to access these maps to support model-based decision-making, but it has recently been proposed that its critical contribution may be instead to integrate information into existing and new models. We tested between these alternatives using an outcome-specific devaluation task and a high-potency chemogenetic approach. We found that selectively inactivating OFC principal neurons when rats learned distinct cue-outcome associations, but prior to outcome devaluation, disrupted subsequent model-based inference, confirming that the OFC is critical for creating new cognitive maps. However, OFC inactivation surprisingly led to generalized devaluation. Using a novel reinforcement learning framework, we demonstrate that this effect is best explained not by a switch to a model-free system, as would be traditionally assumed, but rather by a circumscribed deficit in defining credit assignment precision during model construction. We conclude that the critical contribution of the OFC to learning is regulating the specificity of associations that comprise cognitive maps. One Sentence SummaryOFC inactivation impairs learning of new specific cue-outcome associations without disrupting model-based learning in general.

neuroscience↗

bioRxiv2021

Optimism and Pessimism in Optimised Replay

The replay of task-relevant trajectories is known to contribute to memory consolidation and improved task performance. A wide variety of experimental data show that the content of replayed sequences is highly specific and can be modulated by reward as well as other prominent task variables. However, the rules governing the choice of sequences to be replayed still remain poorly understood. One recent theoretical suggestion is that the prioritization of replay experiences in decision-making problems is based on their effect on the choice of action. We show that this implies that subjects should replay sub-optimal actions that they dysfunctionally choose rather than optimal ones, when, by being forgetful, they experience large amounts of uncertainty in their internal models of the world. We use this to account for recent experimental data demonstrating exactly pessimal replay, fitting model parameters to the individual subjects choices.

neuroscience↗

bioRxiv2021

Metacognitive Computations for Information Search: Confidence in Control

The metacognitive sense of confidence can play a critical role in regulating decisionmaking. In particular, a lack of confidence can justify the explicit, potentially costly, instrumental acquisition of extra information that might resolve the underlying uncertainty. Recent work has suggested a statistically sophisticated tapestry behind the information governing both the making and monitoring of choices. Here, we extend this tapestry to reveal extra richness in the use of confidence for controlling information seeking. We thereby highlight how different models of metacognition can generate diverse relationships between action, confidence, and information search. More broadly, our work shows how crucial it can be to treat metacognitive monitoring and control together.

neuroscience↗

bioRxiv2021

Control over patch encounters changes foraging behaviour

Foraging is a common decision problem in natural environments. When new exploitable sites are always available, a simple optimal strategy is to leave a current site when its return falls below a single average reward rate. Here, we examined foraging in a more structured environment, with a limited number of sites that replenished at different rates and had to be revisited. When participants could choose sites, they visited fast-replenishing sites more often, left sites at higher levels of reward, and achieved a higher net reward rate. Decisions to exploit-or-leave a site were best explained with a computational model that included both the average reward rate for the environment and reward information about the unattended sites. This suggests that unattended sites influence leave decisions, in foraging environments where sites can be revisited. HighlightsO_LIBeing able to select sites during foraging increased visits to high value sites C_LIO_LIThis visitation pattern was efficient, producing higher average reward rates C_LIO_LIDecisions to leave a site were influenced by information about alternative sites C_LI Graphical Abstract O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=200 SRC="FIGDIR/small/426950v3_ufig1.gif" ALT="Figure 1"> View larger version (37K): org.highwire.dtl.DTLVardef@2e662org.highwire.dtl.DTLVardef@6352eaorg.highwire.dtl.DTLVardef@d8aae7org.highwire.dtl.DTLVardef@bc60a6_HPS_FORMAT_FIGEXP M_FIG C_FIG

animal behavior and cognition↗

bioRxiv2021

Dopamine enhances model-free credit assignment through boosting of retrospective model-based inference

Dopamine is implicated in signalling model-free (MF) reward prediction errors and various aspects of model-based (MB) credit assignment and choice. Recently, we showed that cooperative interactions between MB and MF systems include guidance of MF credit assignment by MB inference. Here, we used a double-blind, placebo-controlled, within-subjects design to test the hypothesis that enhancing dopamine levels, using levodopa, boosts the guidance of MF credit assignment by MB inference. We found that levodopa enhanced retrospective guidance of MF credit assignment by MB inference, without impacting on MF and MB influences per se. This drug effect positively correlated with working memory, but only in a context where reward needed to be recalled for MF credit assignment. The dopaminergic enhancement in MB-MF interactions correlated negatively with a dopamine-dependent change in MB credit assignment, possibly reflecting a potential trade-off between these two components of behavioural control. Thus, our findings demonstrate that dopamine boosts MB inference during guidance of MF learning, supported in part by working memory, but trading-off with a dopaminergic enhancement of MB credit assignment. The findings highlight a novel role for a DA influence on MB-MF interactions.

neuroscience↗

bioRxiv2020

Neural encoding of socially adjusted value during competitive and hazardous foraging

In group foraging organisms, optimizing the conflicting demands of competitive food loss and safety is critical. We demonstrate that humans select competition avoidant and risk diluting strategies during foraging depending on socially adjusted value. We formulate a mathematically grounded quantification of socially adjusted value in foraging environments and show using multivariate fMRI analyses that socially adjusted value is encoded by mid-cingulate and ventromedial prefrontal cortices, regions that integrate value and action signals.

neuroscience↗