Biology subjects

Dayan, P.

Publications and source records attributed to Dayan, P..

At least 55 records · Page 3Linked to original sources

bioRxiv2020

Neural Network Poisson Models for Behavioural and Neural Spike Train Data

AO_SCPLOWBSTRACTC_SCPLOWOne of the most important and challenging application areas for complex machine learning methods is to predict, characterize and model rich, multi-dimensional, neural data. Recent advances in neural recording techniques have made it possible to monitor the activities of a large number of neurons across different brain regions as animals perform behavioural tasks. This poses the critical challenge of establishing links between neural activity at a microscopic scale, which might for instance represent sensory input, and at a macroscopic scale, which then generates behaviour. Predominant modeling methods apply rather disjoint techniques to these scales; by contrast, we suggest an end-to-end model which exploits recent developments of flexible, but tractable, neural network point-process models to characterize dependencies between stimuli, actions, and neural data. We apply this model to a public dataset collected using Neuropixel probes in mice performing a visually-guided behavioural task as well as a synthetic dataset produced from a hierarchical network model with reciprocally connected sensory and integration circuits intended to characterize animal behaviour in a fixed-duration motion discrimination task. We show that our model outperforms previous approaches and contributes novel insights into the relationships between neural activities and behaviour.

neuroscience↗

bioRxiv2020

Subjective Beliefs In, Out, and About Control: A Quantitative Analysis

A critical facet of adjusting one’s behaviour after succeeding or failing at a task is assigning responsibility for the ultimate outcome. Humans have trait- and state-like tendencies to implicate aspects of their own behaviour (called ‘internal’ ascriptions) or facets of the particular task or Lady Luck (‘chance’). However, how these tendencies interact with actual performance is unclear. We designed a novel task in which subjects had to learn the likelihood of achieving their goals, and the extent to which this depended on their efforts. High internality (Levenson I-score) was associated with decision making patterns that are less vulnerable to failure, and at the same time less oriented to more rewarding achievements. Our computational analyses suggested that this depended heavily on the adjustment in the perceived achievability of riskier goals following failure. We found beliefs about chance not to be explanatory of choice behaviour in our task. Beliefs about powerful others were strong predictors of behaviour, but only when subjects lacked substantial influence over the outcome. Our results provide an evidentiary basis for heuristics and learning differences that underlie the formation and maintenance of control expectations by the self.Competing Interest StatementThe authors have declared no competing interest.View Full Text

animal behavior and cognition↗

bioRxiv2020

A Comparison of Pruning During Multi-Step Planning in Depressed and Healthy Individuals

BackgroundReal-life decisions are often complex because they involve making sequential choices that constrain future options. We have previously shown that to render such multi-step decisions manageable, people "prune" (i.e. selectively disregard) branches of decision trees that contain negative outcomes. We have theorized that sub-optimal pruning contributes to depression by promoting an oversampling of branches that result in unsavoury outcomes, which results in a negatively-biased valuation of the world. However, no study has tested this theory in depressed individuals. MethodsThirty unmedicated depressed and 31 healthy participants were administered a sequential reinforcement-based decision-making task to determine pruning behaviours, and completed measures of depression and anxiety. Computational, Bayesian and frequentist analyses examined group differences in task performance and relationships between pruning and depressive symptoms. ResultsConsistent with prior findings, participants robustly pruned branches of decision trees that began with large losses, regardless of the potential utility of those branches. However, there was no group difference in pruning behaviours. Further, there was no relationship between pruning and levels of depression/anxiety. LimitationsThe relatively small sample size limited the examination of individual differences. The use of other heuristics that are used to render complex decisions feasible, such as memoization and fragmentation, were not examined. ConclusionsWe found no evidence that sub-optimal pruning is evident in depression. Future research could determine whether maladaptive pruning behaviours are observable in specific sub-groups of depressed patients (e.g. in treatment-resistant individuals), or whether misuse of other heuristics may contribute to depression.

systems biology↗

bioRxiv2020

The roles of online and offline replay in planning

Animals and humans replay neural patterns encoding trajectories through their environment, both whilst they solve decision-making tasks and during rest. Both on-task and off-task replay are believed to contribute to flexible decision making, though how their relative contributions differ remains unclear. We investigated this question by using magnetoencephalography to study human subjects while they performed a decision-making task that was designed to reveal the decision algorithms employed. We characterized subjects in terms of how flexibly each adjusted their choices to changes in temporal, spatial and reward structure. The more flexible a subject, the more they replayed trajectories during task performance, and this replay was coupled with re-planning of the encoded trajectories. The less flexible a subject, the more they replayed previously and subsequently preferred trajectories during rest periods between task epochs. The data suggest that online and offline replay both participate in planning but support distinct decision strategies.

neuroscience↗

bioRxiv2020

Adversarial manipulation of human decision-making

Adversarial examples are carefully crafted input patterns that are surprisingly poorly classified by artificial and/or natural neural networks. Here we examine adversarial vulnerabilities in the processes responsible for learning and choice in humans. Building upon recent recurrent neural network models of choice processes, we propose a general framework for generating adversarial opponents that can shape the choices of individuals in particular decision-making tasks towards the behavioural patterns desired by the adversary. We show the efficacy of the framework through two experiments involving action selection and response inhibition. We further investigate the strategy used by the adversary in order to gain insights into the vulnerabilities of human choice. The framework may find applications across behavioural sciences in helping detect and avoid flawed choice.

neuroscience↗

bioRxiv2019

Combined model-free and model-sensitive reinforcement learning in non-human primates

Contemporary reinforcement learning (RL) theory suggests that potential choices can be evaluated by strategies that may or may not be sensitive to the computational structure of tasks. A paradigmatic model-free (MF) strategy simply repeats actions that have been rewarded in the past; by contrast, model-sensitive (MS) strategies exploit richer information associated with knowledge of task dynamics. MF and MS strategies should typically be combined, because they have complementary statistical and computational strengths; however, this tradeoff between MF/MS RL has mostly only been demonstrated in humans, often with only modest numbers of trials. We trained rhesus monkeys to perform a two-stage decision task designed to elicit and discriminate the use of MF and MS methods. A descriptive analysis of choice behaviour revealed directly that the structure of the task (of MS importance) and the reward history (of MF and MS importance) significantly influenced both choice and response vigour. A detailed, trial-by-trial computational analysis confirmed that choices were made according to a combination of strategies, with a dominant influence of a particular form of model sensitivity that persisted over weeks of testing. The residuals from this model necessitated development of a new combined RL model which incorporates a particular credit assignment weighting procedure. Finally, response vigor exhibited a subtly different collection of MF and MS influences. These results provide new illumination onto RL behavioural processes in non-human primates.

neuroscience↗

bioRxiv2019

Dynamic preferences account for inter-animal variability during the continual learning of a cognitive task

Individual animals perform tasks in different ways, yet the nature and origin of that variability is poorly understood. In the context of spatial memory tasks, variability is often interpreted as resulting from differences in memory ability, but the validity of this interpretation is seldom tested since we lack a systematic approach for identifying and understanding factors that make one animals behavior different than another. Here we identify such factors in the context of spatial alternation in rats, a task often described as relying solely on memory of past choices. We combine hypothesis-driven behavioral design and reinforcement learning modeling to identify spatial preferences that, when combined with memory, support learning of a spatial alternation task. Identifying these preferences allows us to capture differences among animals, including differences in overall learning ability. Our results show that to understand the complexity of behavior requires quantitative accounts of the preferences of each animal.

neuroscience↗

bioRxiv2019

Humans Use Future Thinking to Exert Social Control

Social control, the ability to exert influence over others, is critical in interpersonal interactions yet uninvestigated. Here, we used functional neuroimaging and a social exchange paradigm in which peoples current choices either did, or did not, influence their partners proposals in the future. Computational modeling revealed that participants used forward thinking and calculated the downstream effects of their current actions regardless of the controllability of the social environment. Furthermore, greater levels of estimated control correlated with better performance in controllable interactions and less illusory beliefs about control in uncontrollable interactions. Neural instantiation of trial-by-trial values of social controllability were tracked in the ventromedial prefrontal cortex (vmPFC), striatum, and insula for controllable interactions, but only in vmPFC for uncontrollable interactions. These findings demonstrate that humans use forward thinking, a strategy similar to model-based planning, to guide social choices; and that subjective beliefs about social controllability might not be grounded in reality.

neuroscience↗

bioRxiv2019

Disentangled behavioral representations

Individual characteristics in human decision-making are often quantified by fitting a parametric cognitive model to subjects behavior and then studying differences between them in the associated parameter space. However, these models often fit behavior more poorly than recurrent neural networks (RNNs), which are more flexible and make fewer assumptions about the underlying decision-making processes. Unfortunately, the parameter and latent activity spaces of RNNs are generally high-dimensional and uninterpretable, making it hard to use them to study individual differences. Here, we show how to benefit from the flexibility of RNNs while representing individual differences in a low-dimensional and interpretable space. To achieve this, we propose a novel end-to-end learning framework in which an encoder is trained to map the behavior of subjects into a low-dimensional latent space. These low-dimensional representations are used to generate the parameters of individual RNNs corresponding to the decision-making process of each subject. We introduce terms into the loss function that ensure that the latent dimensions are informative and disentangled, i.e., encouraged to have distinct effects on behavior. This allows them to align with separate facets of individual differences. We illustrate the performance of our framework on synthetic data as well as a dataset including the behavior of patients with psychiatric disorders.

bioinformatics↗

bioRxiv2019

Hippocampal-midbrain circuit enhances the pleasure of anticipation in the prefrontal cortex

Having something to look forward to is a keystone of well-being. Anticipation of a future reward, like an upcoming vacation, can often be more gratifying than the very experience itself. Theories of anticipation have described how it induces behaviors ranging from beneficial information-seeking through to harmful addiction. However, it remains unclear how neural systems compute an attractive value from anticipation, instead of from the reward itself. To address this gap, we administered a decision-making task to human participants that allowed us to analyze brain activity during receipt of information predictive of future pleasant outcomes. Using a computational model of anticipatory value that captures participants decisions, we show that an anticipatory value signal is orchestrated by influences from three brain regions. Ventromedial prefrontal cortex (vmPFC) tracks the value of anticipation; dopaminergic midbrain responds to information that enhances anticipation, while sustained hippocampal activity provides a functional coupling between these regions. This coordinating function of the hippocampus is consistent with its known role in episodic future thinking. Our findings shed new light on the neural underpinnings of anticipations influence over decision-making, while also unifying a range of phenomena associated with risk and time-delay preference.

neuroscience↗

bioRxiv2018

Short-term fasting selectively influences impulsivity in healthy individuals

Previous research has shown that short-term fasting in healthy individuals (HIs) is associated with changes in risky decision-making. The current experiment was designed to examine the influence of short-term fasting in HIs on four types of impulsivity: reflection impulsivity, risky decision-making, delay aversion, and action inhibition. HIs were tested twice, once when fasted for 20 hours, and once when satiated. Participants demonstrated impaired action inhibition when fasted; committing significantly more errors of commission during a food-related Affective Shifting Task. Participants also displayed decreased reflection impulsivity when fasted, opening significantly more boxes during the Information Sampling Task (IST). There were no significant differences in performance between fasted and satiated sessions for risky decision-making or delay aversion. These findings may have implications for understanding eating disorders such as Bulimia Nervosa (BN). Although BN has been characterised as a disorder of poor impulse control, inconsistent findings when comparing individuals with BN and HIs on behavioural measures of impulsivity question this characterisation. Since individuals with BN undergo periods of short-term fasting, the inconsistent findings could be due to differences in the levels of satiation of participants. The current results indicate that fasting can selectively influence performance on the IST, a measure of impulsivity previously studied in BN. However, the results from the IST were contrary to the original hypothesis and should be replicated before specific conclusions can be made.

neuroscience↗

bioRxiv2018

Phasic norepinephrine is a neural interrupt signal for unexpected events in rapidly unfolding sensory sequences - evidence from pupillometry

The ability to track the statistics of our surroundings is a key computational challenge. A prominent theory (Dayan & Yu, 2006) proposes that the brain monitors for unexpected uncertainty - events which deviate substantially from model predictions, indicating model failure. Norepinephrine (NE) is thought to play a key role in this process by serving as an interrupt signal, initiating model-resetting. However, evidence is from paradigms where participants actively monitored stimulus statistics. To determine whether NE routinely reports the statistical structure of our surroundings, even when not behaviourally relevant, we used rapid tone-pip sequences that contained perceptually salient pattern-changes associated with abrupt structural violations vs. emergence of regular structure. Phasic pupil dilations (PDR) were monitored to assess NE. We reveal a remarkable specificity: When not behaviourally relevant, only abrupt structural violations evoked a PDR. The results demonstrate that NE tracks unexpected uncertainty on rapid time scales relevant to sensory signals.

neuroscience↗

bioRxiv2018

Forgetful inference in a sophisticated world model

Humans and other animals are able to discover underlying statistical structure in their environments and exploit it to achieve efficient and effective performance. However, such structure is often difficult to learn and use because it is obscure, involving long-range temporal dependencies. Here, we analysed behavioural data from an extended experiment with rats, showing that the subjects learned the underlying statistical structure, albeit suffering at times from immediate inferential imperfections as to their current state within it. We accounted for their behaviour using a Hidden Markov Model, in which recent observations are integrated with the recollections of an imperfect memory. We found that over the course of training, subjects came to track their progress through the task more accurately, a change that our model largely attributed to decreased forgetting. This learning to remember decreased reliance on recent observations, which may be misleading, in favour of a longer-term memory.\n\nAuthor summaryHumans and other animals possess the remarkable ability to find and exploit patterns and structures in their experience of a complex and varied world. However, such structures are often temporally extended and latent or hidden, being only partially correlated with immediate observations of the world. This makes it essential to integrate current and historical information, and creates a challenging statistical and computational problem.\n\nHere, we examine the behaviour of rats facing a version of this challenge posed by a brain-stimulation reward task. We find that subjects learned the general structure of the task, but struggled when immediate observations were misleading. We captured this behaviour with a model in which subjects integrated evidence from their observations together with a memory whose imperfections accounted for their errors.\n\nThe subjects performance improved markedly over successive sessions, allowing them to overcome misleading observations. According to the model, this arose from a process of learning to remember in which subjects became better at employing more reliable past observations to determine the hidden state of the world.

neuroscience↗

bioRxiv2018

Neural signatures of detours, shortcuts and back-tracking during navigation

Central to the concept of the cognitive map is that it confers behavioural flexibility, allowing animals to take efficient detours, exploit shortcuts and realise the need to back-track rather than persevere on a poorly chosen route. The neural underpinnings of such naturalistic and flexible behaviour remain unclear. During fMRI we tested human subjects on their ability to navigate to a set of goal locations in a virtual desert island riven by lava, which occasionally shifted to block selected paths (necessitating detours) or receded to open new paths (affording shortcuts). We found that during self-initiated back-tracking, activity increased in frontal regions and the dorsal anterior cingulate cortex, while activity in regions associated with the core default-mode network was suppressed. Detours activated a network of frontal regions compared to shortcuts. Activity in right dorsolateral prefrontal cortex specifically increased when participants encountered new plausible shortcuts but which in fact added to the path (false shortcuts). These results help inform current models as to how the brain supports navigation and planning in dynamic environments.\n\nSignificance StatementAdaptation to change is important for survival. Although real-world spatial environments are prone to continual change, little is known about how the brain supports navigation in dynamic environments where flexible adjustments to route plans are needed. Here, we used fMRI to examine the brain activity elicited when humans took forced detours, identified shortcuts and spontaneously back-tracked along their recent path. Both externally and internally generated changes in the route activated the fronto-parietal attention network, whereas only internally generated changes generated increased activity in the dorsal anterior cingulate cortex with a concomitant disengagement in regions associated with the default-mode network. The results provide new insights into how the brain plans and re-plans in the face of a changing environment.

neuroscience↗

bioRxiv2018

Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models

Neuroscience studies of human decision-making abilities commonly involve sub-jects completing a decision-making task while BOLD signals are recorded using fMRI. Hypotheses are tested about which brain regions mediate the effect of past experience, such as rewards, on future actions. One standard approach to this is model-based fMRI data analysis, in which a model is fitted to the behavioral data, i.e., a subjects choices, and then the neural data are parsed to find brain regions whose BOLD signals are related to the models internal signals. However, the internal mechanics of such purely behavioral models are not constrained by the neural data, and therefore might miss or mischaracterize aspects of the brain. To address this limitation, we introduce a new method using recurrent neural network models that are flexible enough to be jointly fitted to the behavioral and neural data. We trained a model so that its internal states were suitably related to neural activity during the task, while at the same time its output predicted the next action a subject would execute. We then used the fitted model to create a novel visualization of the relationship between the activity in brain regions at different times following a reward and the choices the subject subsequently made. Finally, we validated our method using a previously published dataset. We found that the model was able to recover the underlying neural substrates that were discovered by explicit model engineering in the previous work, and also derived new results regarding the temporal pattern of brain activity.

neuroscience↗

bioRxiv2018

Spotting the path that leads nowhere: Modulation of human theta and alpha oscillations induced by trajectory changes during navigation

The capacity to take efficient detours and exploit novel shortcuts during navigation is thought to be supported by a cognitive map of the environment. Despite advances in understanding the neural basis of the cognitive map, little is known about the neural dynamics associated with detours and shortcuts. Here, we recorded magnetoencephalography from humans as they navigated a virtual desert island riven by shifting lava flows. The task probed their ability to take efficient detours and shortcuts to remembered goals. We report modulation in event-related fields and theta power as participants identified real shortcuts and differentiated these from false shortcuts that led along suboptimal paths. Additionally, we found that a decrease in alpha power preceded back-tracking where participants spontaneously turned back along a previous path. These findings help advance our understanding of the fine-grained temporal dynamics of human brain activity during navigation and support the development of models of brain networks that support navigation.

neuroscience↗

bioRxiv2018

Models that learn how humans learn: the case of depression and bipolar disorders

Computational models of learning and decision-making processes in the brain play an important role in many domains. Such models typically have a constrained structure and make specific assumptions about the underlying human learning processes; these may make them underfit observed behaviours. Here we suggest an alternative method based on learning-to-learn approaches, using recurrent neural networks (RNNs) as a flexible family of models that have sufficient capacity to represent the complex learning and decision-making strategies used by humans. In this approach, an RNN is trained to predict the next action that a subject will take in a decision-making task, and in this way, learns to imitate the processes underlying subjects choices and their learning abilities. We demonstrate the benefits of this approach with a new dataset containing behaviour of uni-polar depression (n=34), bipolar (n=33) and control (n=34) participants in a two-armed bandit task. The results indicate that the new approach is better than baseline reinforcement-learning methods in terms of overall performance and its capacity to predict subjects choices. We show that the model can be interpreted using off-policy simulations, and thereby provide a novel clustering of subjects learning processes - something that often eludes traditional approaches to modelling and behavioural analysis.

bioinformatics↗

bioRxiv2018

Noradrenaline modulates decision urgency during sequential information gathering

Arbitrating between timely choice and extended information gathering is critical in effective decision making. Aberrant information gathering behaviour is said to be a feature of psychiatric disorders such as schizophrenia and obsessive-compulsive disorder. We know little about the neurocognitive control mechanisms that drive such information gathering. In a double-blind placebo-controlled drug study with 60 healthy humans (30 female), we examined the effects of noradrenaline and dopamine antagonism on information gathering. We show that modulating noradrenaline function with propranolol leads to decreased information gathering behaviour and this contrasts with no effect following a modulation of dopamine function. Using a Bayesian computational model, we show sampling behaviour is best explained when including an urgency signal that promotes commitment to an early decision. We demonstrate that noradrenaline blockade promotes the expression of this decision-related urgency signal during information gathering. We discuss the findings with respect to psychopathological conditions that are linked to aberrant information gathering.\n\nSignificance StatementKnowing when to stop gathering information and commit to an option is non-trivial. This is an important element in arbitrating between information gain and energy conservation. In this double-blind, placebo-controlled drug study, we investigated to role of catecholamines noradrenaline and dopamine on sequential information gathering. We found that blocking noradrenaline led to a decrease in information gathering, with no effect seen following dopamine blockade. Using a Bayesian computational model, we show that this noradrenaline effect is driven by an increased decision urgency, a signal that reflects an escalating subjective cost of sampling. The observation that noradrenaline modulates decision urgency suggests new avenues for treating patients that show information gathering deficits.

neuroscience↗