Biology subjects

O'Doherty, J. P.

Publications and source records attributed to O'Doherty, J. P..

17 recordsLinked to original sources

bioRxiv2026

Choice-driven remapping of action- and stimulus-anchored value in human single neurons

Whether stimulus- and action-based choices rely on a shared neural architecture remains intensely debated. While lesion and fMRI studies show conflicting regional dissociation and overlap, the underlying single neuron substrates remain unknown. We recorded single neuron activity from the human ventromedial prefrontal cortex (vmPFC), anterior cingulate cortex (ACC), pre-supplementary motor area (preSMA), and amygdala during distinct action- and stimulus-based reward tasks. Prior to decision-making, neurons across all regions broadly encode value for both choice domains. However, choice commitment triggers domain segregation across the prefrontal cortex. While ACC and vmPFC neurons flexibly track both chosen actions and stimuli, they exhibit a double dissociation for chosen valuation: ACC neurons selectively maintain action-based values, whereas vmPFC neurons selectively maintain stimulus-based values. Conversely, preSMA neurons invariant to task domain track chosen action identity and value. These findings reveal how prefrontal networks transition from domain-general evaluation to domain-segregated choice execution.

neuroscience↗

bioRxiv2025

Model-based and model-free valuation signals in the human brain vary markedly in their relationship to individual differences in behavioral control

Human action selection under reinforcement is thought to rely on two distinct strategies: model-free and model-based reinforcement learning. While behavior in sequential decision-making tasks often reflects a mixture of both, the neural basis of individual differences in their expression remains unclear. To investigate this, we conducted a large-scale fMRI study with 179 participants performing a variant of the two-step task. Using both cluster-defined subgroups and computational parameter estimates, we found that the ventromedial prefrontal cortex encodes model-based and model-free value signals differently depending on individual strategy use. Model-based value signals were strongly linked to the degree of model-based behavioral reliance, whereas model-free signals appeared regardless of model-free behavioral influence. Leveraging the large sample, we found individuals lacking both model-based behavior and model-based neural signals exhibited impaired state prediction errors, suggesting a difficulty in building or updating their internal model of the environment. These findings indicate that model-free signals are ubiquitous across individuals, even in those not behaviorally relying on model-free strategies, while model-based representations appear only in those individuals utilizing such a strategy at the behavioral level, the absence of which may depend in part on underlying difficulties in forming accurate model-based predictions.

neuroscience↗

bioRxiv2025

Is the whole the sum of its parts? Neural computation of consumer bundle valuation in humans

Humans are often tasked with making decisions about bundles of multiple items and very little is known about how the human brain aggregates, computes and represents value in such cases. We investigated how the human brain evaluates consumer items, both individually and in bundles, and how this activity relates to choice behavior. Participants underwent a deep-fMRI scanning protocol while we elicited behavioral valuations for single and bundled items. Behaviorally, we find that bundle values are sub-additively discounted compared to the sum of individual item values. Neurally, we find that the same distributed network in pre-frontal cortex computes the value of a bundle and its constituent individual items, but the value representation undergoes a normalization that actively re-scales across bundle and single item contexts. These findings suggest that generalized value regions contextually adapt within a valuation hierarchy, as opposed to utilizing an absolute value code.

neuroscience↗

bioRxiv2025

The representation and valuation of subgoals in the human brain during model-based hierarchical behavior

The human capacity to plan and perform long, complex sequences of behavior to achieve distant goals depends in part on a hierarchical organization that divides behavior into structured segments. Such a mechanism requires the internal designation of certain states as subgoals to mark the successful implementation of a behavioral segment. How the brain represents subgoals over time and computes decision values as a function of subgoals is unknown. While most characterizations of hierarchical behavior lack knowledge of the environment, human decision-making also relies on planning with an internal model of the world. Consequently, it remains to be determined how the brain computes values of subgoals using model-based planning in order to drive hierarchical, model-based decisions. Using a sequential-subgoal decision-making task designed to evoke hierarchical, model-based behavior in combination with fMRI, we decoded a representation of the current subgoal in insula and ventromedial prefrontal cortex during decision-making that persisted over time-a critical, latent representation for computing values and orienting behavior in the correct sequence. Using a model-based, hierarchical reinforcement learning model, we also found key decision signals based on values from the model in several regions of frontal cortex. These findings thereby shed light on the neural correlates of subgoal representation and illustrate how value signals can be computed on the basis of these subgoals and knowledge of the environment structure.

neuroscience↗

bioRxiv2024

Computational and neural evidence for altered fast and slow learning from losses in gambling disorder

Learning occurs across multiple timescales, with fast learning crucial for adapting to sudden environmental changes, and slow learning beneficial for extracting robust knowledge from multiple events. Here we asked if miscalibrated fast vs slow learning can lead to maladaptive decision-making in individuals with gambling disorder. Participants with problem gambling and a recreational gambling control group without any symptoms associated with problem gambling performed a probabilistic decision-making task involving reward-learning and loss-avoidance while being scanned with fMRI. Problem gamblers showed impaired reversal learning from losses compared to the control group, with excessive dependence on slow timescales and reduced reliance on fast timescales. fMRI data implicated the putamen, an area associated with habit, and medial prefrontal cortex (PFC) in slow loss-value encoding, with significantly more robust encoding in medial PFC in problem gamblers compared to controls. Problem gamblers also exhibited stronger loss prediction error encoding in the insular cortex. These findings suggest that individuals with problem gambling have an impaired ability to adjust their predictions following losses, manifested by a stronger influence of slow value learning. This impairment could contribute to the behavioral inflexibility of problem gamblers, particularly the persistence in gambling behavior typically observed in those individuals after incurring loss outcomes.

neuroscience↗

bioRxiv2023

Computational and neural mechanisms underlying the influence of action affordances on value-based choice

When encountering a novel situation, an intelligent agent needs to find out which actions are most beneficial for interacting with that environment. One purported mechanism for narrowing down the scope of possible actions is the concept of action affordance. Here, we delve into the neuro-computational mechanisms accounting for how action affordance shapes value-based learning in a novel environment by utilizing a novel task alongside computational modeling of behavioral and fMRI data collected in humans. Our findings indicate that rather than simply exerting an initial or persistent bias on value-driven choices, action affordance is better conceived of as an independent system that concurrently guides action-selection alongside value-based decision-making. These two systems engage in a competitive process to determine final action selection, governed by a dynamic meta controller. We find that the pre-supplementary motor area and anterior cingulate cortex plays a central role in exerting meta-control over the two systems while the posterior parietal cortex integrates the predictions from these two controllers of what action to select, so that the action-selection process dynamically takes into account both the expected value and appropriateness of particular actions for a given scenario.

neuroscience↗

bioRxiv2023

Temporally organized representations of reward and risk in the human brain

AO_SCPLOWBSTRACTC_SCPLOWThe value and uncertainty associated with choice alternatives constitute critical features along which decisions are made. While the neural substrates supporting reward and risk processing have been investigated, the temporal organization by which these computations are encoded remains elusive. Here we leverage the high spatiotemporal precision of intracranial electroencephalography (iEEG) to uncover how representations of decision-related computations unfold in time. We present evidence of locally distributed representations of reward and risk variables that are temporally organized across multiple regions of interest. Reward outcome representations across wide-spread regions follow a temporally cascading order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We highlight the role of the anterior insula in generalizing between reward prediction error (RePE) and risk prediction error (RiPE), within which the encoding of RePE in the distributed iEEG signal predicts RiPE. Together our results emphasize the utility of uncovering temporal dynamics in the human brain for understanding how computational processes critical for value-based decisions under uncertainty unfold.

neuroscience↗

bioRxiv2023

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans

Pavlovian learning depends on multiple and parallel associations leading to distinct classes of conditioned responses that vary in their flexibility following changes in the value of an associated outcome. Here, we aimed to differentiate brain areas involved in learning and encoding associations that are sensitive to changes in the value of an outcome from those that are not sensitive to such changes. To address this question, we combined a Pavlovian learning task with outcome devaluation, eye-tracking and functional magnetic resonance imaging. We used computational modeling to identify brain regions involved in learning stimulus-reward associations and stimulus-stimulus associations, by testing for brain areas correlating with reward-prediction errors and state-prediction errors, respectively. We found that, contrary to theoretical predictions about reward prediction errors being exclusively model-free, voxels correlating with reward prediction errors in the ventral striatum and subgenual anterior cingulate cortex were sensitive to devaluation. On the other hand, brain areas correlating with state prediction errors were found to be devaluation insensitive. In a supplementary analysis, we distinguished brain regions encoding predictions about outcome taste identity from those involved in encoding predictions about its expected spatial location. A subset of regions involved in taste identity predictions were devaluation sensitive while those involved in encoding predictions about spatial location were devaluation insensitive. These findings provide insights into the role of multiple associative mechanisms in the brain in mediating Pavlovian conditioned behavior - illustrating how distinct neural pathways can in parallel produce both devaluation sensitive and devaluation insensitive behaviors.

neuroscience↗

bioRxiv2022

Characterizing habit learning in the human brain at the individual and group levels: a multi-modal MRI study

The dual-process theory of action control postulates that there are two competitive and complementary mechanisms that control our behavior: a goal-directed system that executes deliberate actions, explicitly aimed toward a particular outcome, and a habitual system that autonomously execute well-learned actions, typically following an encounter with a previously associated cue. In line with dual-process theory, animal studies have provided convincing evidence for dissociable neural mechanisms, mainly manifested in cortico-striatal regions, involved in goal-directed and habitual action control. While substantial progress has been made in characterizing the neural mechanism underlying habit learning in animals, we still lack knowledge on how habits are formed and maintained in the human brain. Thus far only one study, conducted more than a decade ago by Tricomi et al. (2009), has been able to induce habitual behavior in humans via extensive training. This study also implicated the posterior putamen in the process, using functional magnetic resonance imaging (fMRI). However, recent attempts to replicate the behavioral results of this study were not successful. This leaves the research of human habits, and particularly the research of their formation through extensive repetition, as well as their neural basis, limited and far behind the animal research in the field. This motivated us to (1) attempt to replicate the behavioral and imaging main findings of Tricomi et al., (2) identify further functional and microstructural neural modifications associated with habit formation and manifestation, and (3) investigate the relationships between functional and structural plasticity and individual differences in habit expression. To this end, in this registered report we used Tricomi et al.s free-operant task along with multi-modal MRI methods in a well-powered sample (n=123). In this task participants sensitivity to outcome devaluation (an index of goal-directed/habitual action control) is tested following either short or extensive training. In contrast to our hypothesis, we were not able to demonstrate habit formation as a function of training duration nor were we able to relate any functional or microstructural plasticity in the putamen with individual habit expression. We found that a pattern of increased activations in the left head of caudate that re-occurred across each days training is associated with goal directed behavior and that increased processing of devalued cues in low-level visual regions was indicative of goal-directed behavior (and vice versa). In a follow-up exploratory analysis comparing habitual and goal-directed subgroups within each experimental group, we found that elevated activations in frontoparietal regions during early stages of training, as well as increased reactivity towards still-valued cues in somatosensory and superior parietal regions were found in individuals that were more inclined to perform goal-directed behavior (compared with more habitual individuals). Taken together, regions commonly implicated in goal-directed behavior were most predictive of individual habit expression. Finally, we also found that differential patterns of training-related microstructural plasticity, as measured with diffusion MRI, in midbrain dopaminergic regions were associated with habit expression. This work provides new insights into the neural dynamics involved in individual habit formation/expression and encourages the development and testing of new, more sensitive, procedures for experimental habit induction in humans.

neuroscience↗

bioRxiv2022

Can the brain strategically go on automatic pilot? An fMRI study investigating the effect of if-then planning on behavioral flexibility

People often have good intentions but fail to adhere to them. Implementation intentions, a form of strategic planning, can help people to close this intention-behavior gap. Their effectiveness has been proposed to depend on the mental formation of a stimulus-response association between a trigger and target behavior, thereby creating an instant habit. If implementation intentions do indeed lead to reliance on habitual control, then this may come at the cost of reduced behavioral flexibility. Furthermore, we would expect a shift from recruitment of corticostriatal brain regions implicated in goal-directed control towards habit regions. To test these ideas, we performed a functional MRI study in which participants received instrumental training supported by either implementation or goal intentions, followed by an outcome-revaluation to test reliance on habitual versus goal-directed control. We found that implementation intentions led to increased efficiency during training, as reflected in higher accuracy, faster reaction times, and decreased engagement of the anterior caudate. However, implementation intentions did not reduce behavioral flexibility when goals changed during the test phase, nor did it affect the underlying corticostriatal pathways. Additionally, this study showed that slips of action towards devalued outcomes are associated with reduced activity in brain regions implicated in goal-directed control. In conclusion, our behavioral and neuroimaging findings suggest that strategic if-then planning does not lead to a shift from goal-directed towards habitual control.

neuroscience↗

bioRxiv2021

Neurons in human pre-supplementary motor area encode key computations for value-based choice

Adaptive behavior in real-world environments demands that choices integrate over several variables, including the novelty of the options under consideration, their expected value, and uncertainty in value estimation. We recorded neurons from the human pre-supplementary motor area (preSMA), ventromedial prefrontal cortex (vmPFC) and dorsal anterior cingulate to probe how integration over decision variables occurs during decision-making. In contrast to the other areas, preSMA neurons not only represented separate pre-decision variables for each choice option, but also encoded an integrated utility signal and, subsequently, the decision itself. Conversely, post-decision related encoding of variables for the chosen option was more widely distributed and especially prominent in vmPFC. Our findings position the human preSMA as central to the implementation of value-based decisions.

neuroscience↗

bioRxiv2021

Novelty and uncertainty interact to regulate the balance between exploration and exploitation in the human brain.

Recent evidence suggests that both novelty and uncertainty act as potent features guiding exploration. However, these variables are often conflated with each other experimentally, and an understanding of how these attributes interact to regulate the balance between exploration and exploitation has proved elusive. Using a novel task designed to decouple stimulus novelty and estimation uncertainty, we identify separable behavioral and neural mechanisms by which exploration is colored. We show that uncertainty was avoided except when the information gained through exploration could be reliably exploited in the future. In contrast, and contrary to existing theory, novel options grew increasingly attractive relative to familiar counterparts irrespective of the opportunity to leverage their consequences and despite the uncertainty inherent to novel options. These findings led us to develop a formal computational framework in which uncertainty directed choice adapts to the prospective utility of exploration, while novel stimuli persistently draw favor as a result of inflated reward expectations biasing an exploitative strategy. Crucially, novelty is proposed to actively modulate uncertainty processing, effectively blunting the influence of uncertainty in shaping the subjective utility ascribed to novel stimuli. Both behavioral data and fMRI activity sampled from the ventromedial prefrontal cortex, frontopolar cortex and ventral striatum validate this model, thereby establishing a computational account that can not only explain behavior but also shed light on the functional contribution of these key brain regions to the exploration/exploitation trade-off. Our results point to multiple strategies and neural substrates charged with balancing the explore/exploit dilemma, with each targeting distinct aspects of the decision problem to foster a manageable decomposition of an otherwise intractable task.

neuroscience↗

bioRxiv2020

Aesthetic preference for art emerges from a weighted integration over hierarchically structured visual features in the brain

It is an open question whether preferences for visual art can be lawfully predicted from the basic constituent elements of a visual image. Moreover, little is known about how such preferences are actually constructed in the brain. Here we developed and tested a computational framework to gain an understanding of how the human brain constructs aesthetic value. We show that it is possible to explain human preferences for a piece of art based on an analysis of features present in the image. This was achieved by analyzing the visual properties of drawings and photographs by multiple means, ranging from image statistics extracted by computer vision tools, subjective human ratings about attributes, to a deep convolutional neural network. Crucially, it is possible to predict subjective value ratings not only within but also across individuals, speaking to the possibility that much of the variance in human visual preference is shared across individuals. Neuroimaging data revealed that preference computations occur in the brain by means of a graded hierarchical representation of lower and higher level features in the visual system. These features are in turn integrated to compute an overall subjective preference in the parietal and prefrontal cortex. Our findings suggest that rather than being idiosyncratic, human preferences for art can be explained at least in part as a product of a systematic neural integration over underlying visual features of an image. This work not only advances our understanding of the brain-wide computations underlying value construction but also brings new mechanistic insights to the study of visual aesthetics and art appreciation.

neuroscience↗

bioRxiv2019

Value-related neuronal responses in the human amygdala during observational learning

AbstractThe amygdala plays an important role in many aspects of social-cognition and reward-learning. Here we aimed to determine whether human amygdala neurons are involved in the computations necessary to implement learning through observation. We performed single-neuron recordings from the amygdalae of human neurosurgical patients (male and female) while they learned about the value of stimuli through observing the outcomes experienced by another agent interacting with those stimuli. We used a detailed computational modeling approach to describe patients behavior in the task. Then, using both population-level decoding and single neuron analyses we found evidence to implicate amygdala neurons in two key computations relevant for observational-learning: tracking the expected future reward associated with a given stimulus, and in tracking outcome values received by oneself or other agents. Encoding and decoding analyses suggested observational value coding in amygdala neurons occurred in a different subset of neurons than experiential value coding. Collectively, these findings support a key role for the human amygdala in the computations underlying the capacity for learning through observation. Significance statementSingle neuron studies of the human brain provide a unique window into the computational mechanisms of cognition. In this study, epilepsy patients implanted intracranially with depth microelectrodes performed an observational learning task. We measured activity bilaterally in the amygdala and found a representation for observational rewards as well as observational expected reward values. Additionally, the representation of self-experienced and observational values was performed by distinct subsets of amygdala neurons. This study thus provides a rare glimpse into the role of human amygdala neurons in social cognition.

neuroscience↗

bioRxiv2019

Neuro-computational account of arbitration between imitation and emulation during human observational learning

In observational learning (OL), organisms learn from observing the behavior of others. There are at least two distinct strategies for OL. Imitation involves learning to repeat the previous actions of other agents, while in emulation, learning proceeds from inferring the goals and intentions of others. While putative neural correlates for these forms of learning have been identified, a fundamental question remains unaddressed: how does the brain decides which strategy to use in a given situation? Here we developed a novel computational model in which arbitration between the strategies is determined by the predictive reliability, such that control over behavior is adaptively weighted toward the strategy with the most reliable prediction. To test the theory, we designed a novel behavioral task in which our experimental manipulations produced dissociable effects on the reliability of the two strategies. Participants performed this task while undergoing fMRI in two independent studies (the second a pre-registered replication of the first). Behavior manifested patterns consistent with both emulation and imitation and flexibly changed between the two strategies as expected from the theory. Computational modelling revealed that behavior was best described by an arbitration model, in which the reliability of the emulation strategy determined the relative weights allocated to behavior for each strategy. Emulation reliability - the models arbitration signal - was encoded in the ventrolateral prefrontal cortex, temporoparietal junction and rostral cingulate cortex. Being replicated across two fMRI studies, these findings suggest a neuro-computational mechanism for allocating control between emulation and imitation during observational learning.

neuroscience↗

bioRxiv2019

Hippocampal-midbrain circuit enhances the pleasure of anticipation in the prefrontal cortex

Having something to look forward to is a keystone of well-being. Anticipation of a future reward, like an upcoming vacation, can often be more gratifying than the very experience itself. Theories of anticipation have described how it induces behaviors ranging from beneficial information-seeking through to harmful addiction. However, it remains unclear how neural systems compute an attractive value from anticipation, instead of from the reward itself. To address this gap, we administered a decision-making task to human participants that allowed us to analyze brain activity during receipt of information predictive of future pleasant outcomes. Using a computational model of anticipatory value that captures participants decisions, we show that an anticipatory value signal is orchestrated by influences from three brain regions. Ventromedial prefrontal cortex (vmPFC) tracks the value of anticipation; dopaminergic midbrain responds to information that enhances anticipation, while sustained hippocampal activity provides a functional coupling between these regions. This coordinating function of the hippocampus is consistent with its known role in episodic future thinking. Our findings shed new light on the neural underpinnings of anticipations influence over decision-making, while also unifying a range of phenomena associated with risk and time-delay preference.

neuroscience↗

bioRxiv2018

Task complexity interacts with state-space uncertainty in the arbitration process between model-based and model-free reinforcement-learning at both behavioral and neural levels

A major open question concerns how the brain governs the allocation of control between two distinct strategies for learning from reinforcement: model-based and model-free reinforcement learning. While there is evidence to suggest that the reliability of the predictions of the two systems is a key variable responsible for the arbitration process, another key variable has remained relatively unexplored: the role of task complexity. By using a combination of novel task design, computational modeling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process between model-based and model-free RL. We found evidence to suggest that task complexity plays a role in influencing the arbitration process alongside state-space uncertainty. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex bilaterally. These findings provide insight into how the inferior prefrontal cortex negotiates the trade-off between model-based and model-free RL in the presence of uncertainty and complexity, and more generally, illustrates how the brain resolves uncertainty and complexity in dynamically changing environments.\n\nSUMMARY OF FINDINGS- Elucidated the role of state-space uncertainty and complexity in model-based and model-free RL.\n\n- Found behavioral and neural evidence for complexity-sensitive prefrontal arbitration.\n\n- High task complexity induces explorative model-based RL.

neuroscience↗