Biology subjects

Mathar, D.

Publications and source records attributed to Mathar, D..

14 recordsLinked to original sources

bioRxiv2026

Individual Differences in Dopaminergic Modulation of Exploration-Exploitation Behaviour

Dopamine (DA) has been implicated in exploration-exploitation behaviour, i.e., exploring novel, potentially better options vs. exploiting known, previously rewarding options. Impairments in this trade-off occur in psychiatric disorders involving DAergic dysfunction, including addiction and schizophrenia. Pharmacological studies revealed a contribution of DA to exploration, but inconsistent findings suggest that interindividual variability in baseline DA may modulate effects. To address this, we investigated the effects of the DA precursor L-DOPA on exploration-exploitation during reinforcement learning in a sample of N = 75 healthy participants (n = 32 women), following a randomised, double-blind, placebo-controlled, pre-registered design (https://osf.io/p2r7u). We assessed whether putative baseline DA markers, including spontaneous eye blink rate, working memory (WM) capacity, and impulsivity, modulated drug effects and probed visual fixation patterns and pupil dilation as markers of exploration. L-DOPA had no overall effect on computational model parameters of random exploration, directed exploration or choice perseveration. WM capacity moderated drug effects on random exploration, with stronger effects at higher WM capacity. Remaining DA proxies showed no credible effects. Pooling the data from male participants with that from an earlier male-only study (Chakroun et al., 2020; total N = 74), L-DOPA increased uncertainty-dependent value weighting and perseveration strength, while decreasing habit updating, indicating a stronger tendency to repeat previous choices and slower decay of their influence over time. No credible drug effects were observed in female participants. Pupil dilation was tonically increased under L-DOPA and scaled with exploration behaviour and prediction error, confirming that pupillometry can index exploration-exploitation dynamics. Visual exploration patterns reflected uncertainty-driven sampling, but were unaffected by L-DOPA. Taken together, results suggest that DAergic modulation of exploration and perseveration behaviour may be contingent on cognitive capacity and sex, rather than exerting uniform effects across individuals.

neuroscience↗

bioRxiv2025

Reduced Model-Based Control in Gambling Disorder Despite Intact Neural Value and Task Structure Representations

Disordered gambling has been linked to impairments in goal-directed (model-based) control and reinforcement learning. Here we investigated the potential neural basis of this impairment using a sequential reinforcement learning task (modified two-step-task), computational modeling, and functional magnetic resonance imaging (fMRI) in individuals exhibiting symptoms of disordered gambling (GD) and matched healthy controls (HC, n=30 per group). Model-agnostic analyses replicated the effects of reduced performance and reduced model-based control in the gambling group, both in terms of choice and response time effects. Computational modeling of choice behavior confirmed that this effect was due to reduced model-based control in the gambling group. Analyses of choices and response times using drift diffusion modeling revealed a more complex pattern, where behavioral impairments in the gambling group were linked to changes across several parameters reflecting drift rate modulation and asymptote, as well as non-decision time. Despite these pronounced behavioral differences, the gambling group exhibited largely intact neural effects related to the task transition structure, reward feedback and trial-to-trial behavioral adjustments. Results are discussed with respect to current neurocomputational models of behavioral dysregulation in disordered gambling.

neuroscience↗

bioRxiv2025

Catecholamine Precursor Modulation of Human Exploration:Evidence From a Large Gender-Balanced Sample

The catecholamine precursor tyrosine has been linked to improved cognitive performance, but investigations into decision-making and reinforcement learning processes known to be under catecholamine control are sparse. We examined the impact of a single dose of Tyrosine (2g) on reinforcement learning and exploration in a large (n=63) gender-balanced sample in a within-subjects preregistered study. Reinforcement learning performance was improved under Tyrosine, and computational modeling revealed that this performance increase was due to a stabilization of choice behavior reflected in increased value-driven exploitation. Further non-preregistered modeling analyses confirmed that accounting for higher-order perseveration substantially improved model fit, and substantiated the observation of increased value-driven exploitation under Tyrosine. Furthermore, it revealed a more fine-grained computational impact of Tyrosine, showing attenuated effects of directed exploration and value-independent perseveration. Supplementation with Tyrosine therefore improved reinforcement learning performance by stabilizing choice patterns in the service of optimizing reward accumulation. Results confirm that Tyrosine supplementation modulates specific computational mechanisms thought to be under catecholamine control.

neuroscience↗

bioRxiv2024

Dopamine and temporal discounting: revisiting pharmacology and individual differences

Disorders characterised by changes in dopamine (DA) neurotransmission are often linked to changes in the temporal discounting of future rewards. Likewise, pharmacological manipulations of DA neuro-transmission in healthy individuals modulate temporal discounting, but there is considerable variability in the directionality of reported pharmacological effects, as enhancements and reductions of DA signalling have been linked to both increases and reductions of temporal discounting. This may be due to meaningful individual differences in drug effects and/or false positive findings in small samples. To resolve these inconsistencies, we 1) revisited pharmacological effects of the DA precursor L-DOPA on temporal discounting in a large sample of N = 76 healthy participants (n = 44 male) and 2) examined several putative proxy measures for DA to revisit the role of individual differences in a randomised, double-blind placebo-controlled pre-registered study (https://osf.io/a4k9j/). Replicating previous findings, higher rewards were discounted less (magnitude effect). Computational modelling using hierarchical Bayesian parameter estimation confirmed that the data in both drug conditions were best accounted for by a non-linear temporal discounting drift diffusion model. In line with recent animal and human work, L-DOPA reliably reduced the discount rate with a small effect size, challenging earlier findings in substantially smaller samples. We found no credible evidence for linear or quadratic effects of putative DA proxy measures on model parameters, calling into question the role of these measures in accounting for individual differences in DA drug effects.

neuroscience↗

bioRxiv2023

Signatures of heuristic-based directed exploration in two-step sequential decision task behaviour.

Processes formalized in classic Reinforcement Learning (RL) theory, such as model-based (MB) control and exploration strategies have proven fertile in cognitive and computational neuroscience, as well as computational psychiatry. Dysregulations in MB control and exploration and their neurocomputational underpinnings play a key role across several psychiatric disorders. Yet, computational accounts mostly study these processes in isolation. The current study extended standard hybrid models of a widely-used sequential RL-task (two-step task; TST) employed to measure MB control. We implemented and compared different computational model extensions for this task to quantify potential exploration mechanisms. In two independent data sets spanning two different variants of the task, an extension of a classical hybrid RL model with a heuristic-based exploration mechanism provided the best fit, and revealed a robust positive effect of directed exploration on choice probabilities in stage one of the task. Posterior predictive checks further showed that the extended model reproduced choice patterns present in both data sets. Results are discussed with respect to implications for computational psychiatry and the search for neurocognitive endophenotypes.

neuroscience↗

bioRxiv2022

Appetitive cue exposure increases neural reward responses without modulating temporal discounting

When given a choice, humans and many animals prefer smaller but sooner over larger but later rewards, a tendency referred to as temporal discounting. Alterations in devaluation of future rewards have been reported in a range of maladaptive behaviors and clinical conditions. Although temporal discounting is highly stable over time and testing environments (e.g., laboratory vs. virtual reality), it is partly under contextual control. For example, highly appetitive cues such as erotic images might increase preferences for immediate rewards, although overall evidence remains mixed. Dopaminergic circuit activity and striatal dopamine concentrations are often assumed to drive increases in temporal discounting following appetitive cue-exposure, yet this was never explicitly tested. Here we examined cue-reactivity effects (erotic vs. neutral pictures) on subsequent temporal discounting in a pre-registered within-subjects study in healthy male participants (n=38). Functional magnetic resonance imaging (fMRI) assessed neural cue-reactivity, value-computations and choice-related effects. Preregistered analyses replicated previous findings of value coding in ventromedial prefrontal cortices, striatum and cingulate cortex. Likewise, as hypothesized, lateral prefrontal cortex activity increased during choices of delayed rewards, potentially reflecting cognitive control. As predicted, erotic vs. neutral cue exposure was associated with increased activity in attention and reward circuits. Contrary to our preregistered hypotheses, temporal discounting was largely unaffected by cue exposure. Likewise, cue-reactivity in key areas of the dopaminergic reward circuit (Nacc, VTA) was not significantly associated with changes in behavior. Our results indicate that behavioral effects of erotic cue exposure on temporal discounting might not be as unequivocal as previously thought and raise doubt on the hypothesis of an upregulated dopaminergic ramping mechanism, that might support myopic approach behavior towards immediate rewards.

neuroscience↗

bioRxiv2022

Dopamine regulates decision thresholds in human reinforcement learning.

Dopamine fundamentally contributes to reinforcement learning by encoding prediction errors, deviations of an outcome from expectation. Prediction error coding in dopaminergic regions in human functional neuroimaging studies is well replicated. In contrast, replications of behavioral and neural effects of pharmacological modulations of the dopamine system in human reinforcement learning are scarce. Additionally, dopamine contributes to action selection, but direct evidence and process-specific accounts in human reinforcement learning are lacking. Here we examined dopaminergic mechanisms underlying human reinforcement learning in a within-subjects pharmacological approach in male human volunteers (n=31, within-subjects design; Placebo, 150mg L-dopa, 2mg Haloperidol) in combination with functional magnetic resonance imaging and a stationary reinforcement learning task. We had two aims. First, we aimed to replicate previously reported beneficial effects of L-dopa vs. Haloperidol on reinforcement learning from gains. This replication was not successful. We observed no performance benefit of L-Dopa vs. Haloperidol, and no evidence for alterations in neural prediction error signaling. In contrast, Bayesian analyses provided moderate evidence in favor of the null hypothesis. This unsuccessful replication is likely at least partly due to a number of differences in experimental design. Second, using combined reinforcement learning drift diffusion models, we tested the recent proposal that dopamine contributes to action selection by regulating decision thresholds. Model comparison revealed that the data were best accounted for by a reinforcement learning drift diffusion model with separate learning rates for positive and negative prediction errors. The model accounted for both reductions in RTs and increases in accuracy over the course of learning. The only parameter showing robust drug effects was the boundary separation parameter, which revealed reduced decision thresholds under both L-Dopa and Haloperidol, compared to Placebo, and the degree of threshold reduction accounted for individual differences in RTs between conditions. Results are in line with the idea that striatal dopamine regulates decision thresholds during action selection, and that lower dosages of D2 receptor antagonists increase striatal DA release via an inhibition of autoreceptor-mediated feedback mechanisms.

neuroscience↗

bioRxiv2022

Erotic cue exposure increases physiological arousal, biases choices towards immediate rewards and attenuates model-based reinforcement learning

Computational psychiatry focuses on identifying core cognitive processes that appear altered across a broad range of psychiatric disorders. Temporal discounting of future rewards and model-based control during reinforcement learning have proven as two promising candidates. Despite its trait-like stability, temporal discounting has been suggested to be at least partly under contextual control. For example, highly arousing cues such as erotic pictures were shown to increase discounting, although overall evidence to date remains somewhat mixed. Whether model-based reinforcement learning is similarly affected by arousing cues is unclear. Here we tested cue-reactivity effects (erotic pictures) on subsequent temporal discounting and model-based reinforcement learning in a within-subjects design in n=39 healthy male participants. Self-reported and physiological arousal (cardiac activity and pupil dilation) were assessed before and during cue exposure. Arousal was increased during exposure of erotic vs. neutral cues both on the subjective and autonomic level. Erotic cue exposure nominally increased discounting as reflected by reduced choices of delayed options. Hierarchical drift diffusion modeling (DDM) linked increased discounting to a shift in the starting point bias of evidence accumulation towards immediate options. Model-based control during reinforcement learning was reduced following erotic cues according to model-agnostic analysis. Notably, DDM linked this effect to attenuated forgetting rates of unchosen options, leaving the model-based control parameter unchanged. Our findings replicate previous work on cue-reactivity effects in temporal discounting and for the first time show similar effects in model-based reinforcement learning. Our results highlight how environmental cues can impact core human decision processes and reveal that comprehensive drift diffusion modeling approaches can yield novel insights in reward-based decision processes.

neuroscience↗

bioRxiv2022

A single dose of the catecholamine precursor Tyrosine reduces physiological arousal and decreases decision thresholds in reinforcement learning and temporal discounting

Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, but overall findings are mixed. Here, we investigate the effect of a single dose of tyrosine (2g) vs. placebo on two key aspects of catecholamine-dependent decision-making: model-based reinforcement learning (2-step task) and temporal discounting, using a double-blind, placebo-controlled, within-subject design (n=28 healthy male participants). We leveraged drift diffusion models in a hierarchical Bayesian framework to jointly model participants choices and response times in both tasks. Furthermore, comprehensive autonomic monitoring (heart rate, heart rate variability, pupillometry, spontaneous eye-blink rate) was performed both pre- and post-supplementation, to explore potential physiological effects of supplementation. Across tasks, tyrosine consistently reduced participants RTs without deteriorating task-performance. Diffusion modeling linked this effect to attenuated decision-thresholds in both tasks and further revealed increased model-based control (2-step task) and (if anything) attenuated temporal discounting. On the physiological level, participants pupil dilation was predictive of the individual degree of temporal discounting. Tyrosine supplementation reduced physiological arousal as revealed by increases in pupil dilation variability and reductions in hear rate. Supplementation-related changes in physiological arousal predicted individual changes in temporal discounting. Our findings provide first evidence that tyrosine supplementation might impact psychophysiological parameters, and suggest that modeling approaches based on sequential sampling models can yield novel insights into latent cognitive processes modulated by amino-acid supplementation.

neuroscience↗

bioRxiv2021

Parameter and model recovery of reinforcement learning models for restless bandit problems

Multi-armed restless bandit tasks are regularly applied in psychology and cognitive neuroscience to assess exploration and exploitation behavior in structured environments. These models are also readily applied to examine effects of (virtual) brain lesions on performance, and to infer neurocomputational mechanisms using neuroimaging or pharmacological approaches. However, to infer individual, psychologically meaningful parameters from such data, computational cognitive modeling is typically applied. Recent studies indicate that softmax (SM) decision rule models that include a representation of environmental dynamics (e.g. the Kalman Filter) and additional parameters for modeling exploration and perseveration (Kalman SMEP) fit human bandit task data better than competing models. Parameter and model recovery are two central requirements for computational models: parameter recovery refers to the ability to recover true data-generating parameters; model recovery refers to the ability to correctly identify the true data generating model using model comparison techniques. Here we comprehensively examined parameter and model recovery of the Kalman SMEP model as well as nested model versions, i.e. models without the additional parameters, using simulation and Bayesian inference. Parameter recovery improved with increasing trial numbers, from around .8 for 100 trials to around .93 for 300 trials. Model recovery analyses likewise confirmed acceptable recovery of the Kalman SMEP model. Model recovery was lower for nested Kalman filter models as well as delta rule models with fixed learning rates. Exploratory analyses examined associations of model parameters with model-free performance metrics. Random exploration, captured by the inverse softmax temperature, was associated with lower accuracy and more switches. For the exploration bonus parameter modeling directed exploration, we confirmed an inverse-U-shaped association with accuracy, such that both an excess and a lack of directed exploration reduced accuracy. Taken together, these analyses underline that the Kalman SMEP model fulfills basic requirements of a cognitive model.

neuroscience↗

bioRxiv2021

Increased temporal discounting and reduced model-based control in problem gambling are not substantially modulated by exposure to virtual gambling environments.

High-performance virtual reality (VR) technology has opened new possibilities for the examination of the reactivity towards addiction-related cues (cue-reactivity) in addiction. In this preregistered study (https://osf.io/4mrta), we investigated the subjective, physiological, and behavioral effects of gambling-related VR environment exposure in participants reporting frequent or pathological gambling (n=31) as well as non-gambling controls (n=29). On two separate days, participants explored two rich and navigable VR-environments (neutral: cafe vs. gambling-related: casino/sports-betting facility), while electrodermal activity and heart rate were continuously measured using remote sensors. Within VR, participants performed a temporal discounting task and a sequential decision-making task designed to assess model-based and model-free contributions to behavior. Replicating previous findings, we found strong evidence for increased temporal discounting and reduced model-based control in participants reporting frequent or pathological gambling. Although VR gambling environment exposure increased subjective craving, there was if anything inconclusive evidence for further behavioral or physiological effects. Instead, VR exposure substantially increased physiological arousal (electrodermal activity), across groups and conditions. VR is a promising tool for the investigation of context effects in addiction, but some caution is warranted since effects of real gambling environments might not generally replicate in VR. Future studies should delineate how factors such as cognitive load and ecological validity could be balanced to create a more naturalistic VR experience.

neuroscience↗

bioRxiv2021

Gambling environment exposure increases temporal discounting but improves model-based control in regular slot-machine gamblers.

Gambling disorder is a behavioral addiction that negatively impacts personal finances, work, relationships and mental health. In this pre-registered study (https://osf.io/5ptz9/?view_only=62956a2afcd1495984db4be18c54b50a) we investigated the impact of real-life gambling environments on two computational markers of addiction, temporal discounting and model-based reinforcement learning. Gambling disorder is associated with increased temporal discounting and reduced model-based learning. Regular gamblers (n = 30, DSM-5 score range 3-9) performed both tasks in a neutral (cafe) and a gambling-related environment (slot-machine venue) in counterbalanced order. Data were modeled using drift diffusion models for temporal discounting and reinforcement learning via hierarchical Bayesian estimation. Replicating previous findings, gamblers discounted rewards more steeply in the gambling-related context. This effect was positively correlated with gambling related cognitive distortions (pre-registered analysis). In contrast to our pre-registered hypothesis, model-based reinforcement learning was improved in the gambling context. Here we show that temporal discounting and model-based reinforcement learning are modulated in opposite ways by real-life gambling cue exposure. Results challenge aspects of habit theories of addiction, and reveal that laboratory-based computational markers of psychopathology are under substantial contextual control.

neuroscience↗

bioRxiv2019

Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making

A central issue in reinforcement learning and decision-making is whether to exploit knowledge of reward values, or to explore novel options. Although it is widely hypothesized that dopamine neurotransmission plays a key role in regulating this balance, causal evidence for a role of dopamine in human exploration is still lacking. Here, we use a combination of computational modeling, pharmacological intervention and functional magnetic resonance imaging (fMRI) to test for a causal effect of dopamine transmission on the exploration-exploitation trade-off in humans. 31 healthy male subjects performed a restless four-armed bandit task in a within-subjects design under three drug conditions: 150mg of the dopamine precursor L-dopa, 2mg of the D2 receptor antagonist haloperidol, and placebo. In all conditions, choice behavior was best explained by an extension of an established Bayesian learning model accounting for perseveration, uncertainty-based exploration and random exploration. Uncertainty-based exploration was attenuated under L-dopa compared to placebo and haloperidol. There was no evidence for a modulation of prediction error signaling or categorical effects of exploration/exploitation under L-dopa, whereas model-based fMRI revealed that L-dopa attenuated neural representations of overall uncertainty in insula and dorsal anterior cingulate cortex. Our results highlight the computational role of these regions in exploration and suggest that dopamine modulates exploration by modulating how this circuit tracks accumulating uncertainty during decision-making.

neuroscience↗

bioRxiv2017

A potential link between gambling addiction severity and central dopamine levels -- evidence from spontaneous eye blink rates in gamblers and controls

Accumulating evidence points at similarities between substance use disorders and pathological gambling on the behavioral and neural level. In substance addiction, dysregulation of striatal dopamine transmission has been consistently revealed. Due to the neurotoxicity of stimulating substances, it is still debated if this constitutes mainly a consequence of recurrent substance abuse or a vulnerability marker for addiction disorders. For gambling addiction, no clear association with striatal dopamine levels has been unveiled so far. With its presumably negligible dopaminergic toxicity, possible differences in striatal dopamine transmission in gambling addiction might therefore constitute a vulnerability marker. Spontaneous eye blink rate (sEBR) is controversially discussed as a potential proxy measure for striatal dopamine levels. Here we examined sEBR in 21 male problem gamblers and 20 healthy control participants. In addition, participants completed a screening questionnaire for overall psychopathology and self-reported measures of alcohol and nicotine consumption. We found no significant difference in sEBR between gamblers and controls. However, in gamblers, sEBR was negatively associated with addiction severity and positively associated with psychopathology. A final exploratory analysis revealed that healthy controls with low sEBR displayed higher alcohol and nicotine consumption than healthy participants with high sEBR. Although the association between dopamine transmission and sEBR is still debated, our findings reveal that sEBR is sensitive to inter-individual differences in addiction severity in problem gamblers.

neuroscience↗