Biology subjects

Ganzer, F.

Publications and source records attributed to Ganzer, F..

3 recordsLinked to original sources

bioRxiv2022

Dopamine regulates decision thresholds in human reinforcement learning.

Dopamine fundamentally contributes to reinforcement learning by encoding prediction errors, deviations of an outcome from expectation. Prediction error coding in dopaminergic regions in human functional neuroimaging studies is well replicated. In contrast, replications of behavioral and neural effects of pharmacological modulations of the dopamine system in human reinforcement learning are scarce. Additionally, dopamine contributes to action selection, but direct evidence and process-specific accounts in human reinforcement learning are lacking. Here we examined dopaminergic mechanisms underlying human reinforcement learning in a within-subjects pharmacological approach in male human volunteers (n=31, within-subjects design; Placebo, 150mg L-dopa, 2mg Haloperidol) in combination with functional magnetic resonance imaging and a stationary reinforcement learning task. We had two aims. First, we aimed to replicate previously reported beneficial effects of L-dopa vs. Haloperidol on reinforcement learning from gains. This replication was not successful. We observed no performance benefit of L-Dopa vs. Haloperidol, and no evidence for alterations in neural prediction error signaling. In contrast, Bayesian analyses provided moderate evidence in favor of the null hypothesis. This unsuccessful replication is likely at least partly due to a number of differences in experimental design. Second, using combined reinforcement learning drift diffusion models, we tested the recent proposal that dopamine contributes to action selection by regulating decision thresholds. Model comparison revealed that the data were best accounted for by a reinforcement learning drift diffusion model with separate learning rates for positive and negative prediction errors. The model accounted for both reductions in RTs and increases in accuracy over the course of learning. The only parameter showing robust drug effects was the boundary separation parameter, which revealed reduced decision thresholds under both L-Dopa and Haloperidol, compared to Placebo, and the degree of threshold reduction accounted for individual differences in RTs between conditions. Results are in line with the idea that striatal dopamine regulates decision thresholds during action selection, and that lower dosages of D2 receptor antagonists increase striatal DA release via an inhibition of autoreceptor-mediated feedback mechanisms.

neuroscience↗

bioRxiv2020

Dopaminergic signals in the Nucleus Accumbens, VTA and vmPFC underpin extinction learning from omitted threats

Learning to be safe is central for adaptive behaviour when threats are no longer present. Detecting the absence of an expected threat is key for threat extinction learning and an essential process for the behavioural treatment of anxiety related disorders. One possible mechanism underlying extinction learning is a dopaminergic mismatch signal that encodes the absence of an expected threat. Here we show that such a dopamine-related pathway underlies extinction learning in humans. Dopaminergic enhancement via administration of L-DOPA (vs. Placebo) was associated with reduced retention of differential psychophysiological threat responses at later test, which was mediated by activity in the ventromedial prefrontal cortex that was specific to extinction learning. L-DOPA administration enhanced signals at the time-point of an expected, but omitted threat in extinction learning within the nucleus accumbens, which were functionally coupled with the ventral tegmental area and the amygdala. Computational modelling of threat expectancies further revealed prediction error encoding in nucleus accumbens that was reduced when L-DOPA was administered. Our results thereby provide evidence that extinction learning is influenced by L-DOPA and provide a mechanistic perspective to augment extinction learning by dopaminergic enhancement in humans. Summary COI statementNo conflict of interest for any of the authors. The funder had no role in the design of the study, the collection and analysis of data, or the preparation of the manuscript.

neuroscience↗

bioRxiv2019

Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making

A central issue in reinforcement learning and decision-making is whether to exploit knowledge of reward values, or to explore novel options. Although it is widely hypothesized that dopamine neurotransmission plays a key role in regulating this balance, causal evidence for a role of dopamine in human exploration is still lacking. Here, we use a combination of computational modeling, pharmacological intervention and functional magnetic resonance imaging (fMRI) to test for a causal effect of dopamine transmission on the exploration-exploitation trade-off in humans. 31 healthy male subjects performed a restless four-armed bandit task in a within-subjects design under three drug conditions: 150mg of the dopamine precursor L-dopa, 2mg of the D2 receptor antagonist haloperidol, and placebo. In all conditions, choice behavior was best explained by an extension of an established Bayesian learning model accounting for perseveration, uncertainty-based exploration and random exploration. Uncertainty-based exploration was attenuated under L-dopa compared to placebo and haloperidol. There was no evidence for a modulation of prediction error signaling or categorical effects of exploration/exploitation under L-dopa, whereas model-based fMRI revealed that L-dopa attenuated neural representations of overall uncertainty in insula and dorsal anterior cingulate cortex. Our results highlight the computational role of these regions in exploration and suggest that dopamine modulates exploration by modulating how this circuit tracks accumulating uncertainty during decision-making.

neuroscience↗