Biology subjects

Verbeke, P.

Publications and source records attributed to Verbeke, P..

10 recordsLinked to original sources

bioRxiv2025

Two time scales of adaptation in human learning rates

Different situations may require radically different information updating speeds (i.e., learning rates). Some demand fast learning rates, while others benefit from using slower ones. To adjust learning rates, decision makers could rely on either global, meta-learned differences between environments, or faster but transient adaptations to locally experienced prediction errors. Here, we introduce a new paradigm that allows researchers to measure and empirically disentangle both forms of adaptations. Participants performed short blocks of trials of a continuous estimation task - fishing for crabs - on six different islands that required different optimal (initial) learning rates. Across two experiments, participants showed fast adaptations in learning rate within a block. Critically, participants also learned global environment-specific learning rates over the time course of the experiment, as evidenced by computational modelling and by the learning rates calculated on the very first trial when revisiting an environment (i.e., unconfounded by transient adaptations). Using representational similarity analyses of fMRI data, we found that differences in voxel pattern responses in the central orbitofrontal cortex correlated with differences in these global environment-specific learning rates. Our findings show that humans adapt learning rates at both slow and fast time scales, and that the central orbitofrontal cortex may support meta-learning by representing environment-specific task-relevant features such as learning rates.

neuroscience↗

bioRxiv2025

Neural and computational evidence for a predictive learning account of the testing effect

Testing enhances memory more than studying. Although numerous studies have demonstrated the robustness of this classic effect, its neural and computational origin remains debated. Predictive learning is a potential mechanism behind this phenomenon: Because predictions and prediction errors (mismatch between predictions and feedback) are more likely to be generated in testing (relative to in studying), testing can benefit more from predictive learning. We shed light on the testing effect from a multi-level analysis perspective via a combination of cognitive neuroscience experiments (fMRI) and computational modeling. Behaviorally and computationally, only a model incorporating predictive learning can account for the full breadth of behavioral patterns and the robust testing effect. At the neural level, testing and prediction error both activate the canonical reward-related brain areas in the ventral striatum, insula, and midbrain. Crucially, back sorting analysis revealed that activation in the ventral striatum, insula, and midbrain can enhance declarative memory. These results provide strong and converging evidence for a predictive learning account of the testing effect. Significance StatementAn exam is not a neutral measurement of memory. The testing effect entails that a test (e.g., an exam), is more effective than study for learning and memory. The same effect can be harnessed also before events of significance take place, rendering it an important aspect of an active learning strategy. Nevertheless, its origin remains unknown. We propose a novel predictive learning account, which posits that testing (vs studying) facilitates predictions about study material and promotes learning from prediction errors. Computationally, the testing effect was explained through a predictive-learning-based neural network. Neurally, testing and prediction error activate common neural areas, which in turn enhance declarative memory. This account may extend beyond testing to support active learning.

neuroscience↗

bioRxiv2023

Humans adaptively select different computational strategies in different learning environments.

The Rescorla-Wagner rule remains the most popular tool to describe human behavior in reinforcement learning tasks. Nevertheless, it cannot fit human learning in complex environments. Previous work proposed several hierarchical extensions of this learning rule. However, it remains unclear when a flat (non-hierarchical) versus a hierarchical strategy is adaptive, or when it is implemented by humans. To address this question, current work applies a nested modelling approach to evaluate multiple models in multiple reinforcement learning environments both computationally (which approach performs best) and empirically (which approach fits human data best). We consider ten empirical datasets (N = 407) divided over three reinforcement learning environments. Our results demonstrate that different environments are best solved with different learning strategies; and that humans adaptively select the learning strategy that allows best performance. Specifically, while flat learning fitted best in less complex stable learning environments, humans employed more hierarchically complex models in more complex environments.

neuroscience↗

bioRxiv2022

Modulating hierarchical learning by high-definition transcranial alternating current stimulation at theta frequency

Considerable evidence highlights the dorsolateral prefrontal cortex (DLPFC) as a key region for hierarchical (i.e., multi-level) learning. In a previous electroencephalography (EEG) study we found that the low-level prediction errors (PEs) were encoded by frontal theta oscillations (4-7Hz), centered on right DLPFC (rDLPFC). However, the causal relationship between frontal theta oscillations and hierarchical learning remains poorly understood. To investigate this question, in the current study, participants received theta (6Hz) and sham high-definition transcranial alternating current stimulation (HD-tACS) over the rDLPFC, while performing the probabilistic reversal learning task. Behaviorally, theta tACS induced a significant reduction in accuracy for the stable environment, but not for the volatile environment, relative to the sham condition. Computationally, we implemented a combination of a hierarchical Bayesian learning and a decision model. Theta tACS induced a significant increase in low-level (i.e., probability-level) learning rate and uncertainty of low-level estimation relative to sham condition. Instead, the temperature parameter of the decision model, which represents (inverse) decision noise, was not significantly altered due to theta stimulation. These results indicate that theta frequency may modulate the (low-level) learning rate. Furthermore, environmental features (e.g., its stability) may determine whether learning is optimized as a result.

neuroscience↗

bioRxiv2021

Time-based binding as a solution to and a limitation for flexible cognition

Why cant we keep as many items as we want in working memory? It has long been debated whether this resource limitation is a bug (a downside of our fallible biological system) or instead a feature (an optimal response to a computational problem). We propose that the resource limitation is a consequence of a useful feature. Specifically, we propose that flexible cognition requires time-based binding, and time-based binding necessarily limits the number of (bound) memoranda that can be stored simultaneously. Time-based binding is most naturally instantiated via neural oscillations, for which there exists ample experimental evidence. We report simulations that illustrate this theory and that relate it to empirical data. We also compare the theory to several other (feature and bug) resource theories.

neuroscience↗

bioRxiv2021

Using top-down gating to optimally balance shared versus separated task representations

Human adaptive behavior requires continually learning and performing a wide variety of tasks, often with very little practice. To accomplish this, it is crucial to separate neural representations of different tasks in order to avoid interference. At the same time, sharing neural representations supports generalization and allows faster learning. Therefore, a crucial challenge is to find an optimal balance between shared versus separated representations. Typically, models of human cognition employ top- down modulatory signals to separate task representations, but there exist surprisingly little systematic computational investigations of how such modulation is best implemented. We identify and systematically evaluate two crucial features of modulatory signals. First, top-down input can be processed in an additive or multiplicative manner. Second, the modulatory signals can be adaptive (learned) or non-adaptive (random). We cross these two features, resulting in four modulation networks which are tested on a variety of input datasets and tasks with different degrees of stimulus-action mapping overlap. The multiplicative adaptive modulation network outperforms all other networks in terms of accuracy. Moreover, this network develops hidden units that optimally share representations between tasks. Specifically, different than the binary approach of currently popular latent state models, it exploits partial overlap between tasks.

neuroscience↗

bioRxiv2020

Theta oscillations shift towards optimal frequency for cognitive control

Cognitive control is supported by theta band (4-7Hz) neural oscillations coordinating neural populations for task implementation. Task performance has been shown to depend on theta amplitude but a second critical aspect of theta oscillations, its peak frequency, has mostly been overlooked. Using modelling, behavioral and electrophysiological recordings, we show that theta oscillations adapt to task demands by shifting towards the optimal frequency.

neuroscience↗

bioRxiv2020

Theta and alpha power across fast and slow timescales in cognitive control

Theta and alpha frequency neural oscillations are important for learning and cognitive control, but their exact role has remained obscure. In particular, it is unknown whether they operate at similar timescales, and whether they support different cognitive processes. We recorded EEG in 30 healthy human participants while they performed a learning task containing both novel (block-unique) and repeating stimuli. We investigated behavior and electrophysiology at both fast (i.e., within blocks) and slow (i.e., between blocks) time scales. Behaviorally, both response time and accuracy improved (resp. decrease and increase) over both fast and slow timescales. However, on the spectral level, theta power significantly decreased along the slow timescale, whereas alpha power instead significantly increased along the fast timescale. We thus demonstrate that theta and alpha both play a role during learning, but operate at different timescales. This result poses important empirical constraints for theories on learning, cognitive control, and neural oscillations.

neuroscience↗

bioRxiv2020

Learning to synchronize: midfrontal theta dynamics during reversal learning

In recent years, several hierarchical extensions of well-known learning algorithms have been proposed. For example, when stimulus-action mappings vary across time or context, the brain may learn two or more stimulus-action mappings in separate modules, and additionally (at a hierarchically higher level) learn to appropriately switch between those modules. However, how the brain mechanistically coordinates neural communication to implement such hierarchical learning, remains unknown. Therefore, the current study tests a recent computational model that proposed how midfrontal theta oscillations implement such hierarchical learning via the principle of binding by synchrony (Sync model). More specifically, the Sync model employs bursts at theta frequency to flexibly bind appropriate task modules by synchrony. 64-channel EEG signal was recorded while 27 human subjects (Female: 21, Male: 6) performed a probabilistic reversal learning task. In line with the Sync model, post-feedback theta power showed a linear relationship with negative prediction errors, but not with positive prediction errors. This relationship was especially pronounced for subjects with better behavioral fit (measured via AIC) of the Sync model. Also consistent with Sync model simulations, theta phase-coupling between midfrontal electrodes and temporo-parietal electrodes was stronger after negative feedback. Our data suggest that the brain uses theta power and synchronization for flexibly switching between task rule modules, as is useful for example when multiple stimulus-action mappings must be retained and used. Significance StatementEveryday life requires flexibility in switching between several rules. A key question in understanding this ability is how the brain mechanistically coordinates such switches. The current study tests a recent computational framework (Sync model) that proposed how midfrontal theta oscillations coordinate activity in hierarchically lower task-related areas. In line with predictions of this Sync model, midfrontal theta power was stronger when rule switches were most likely (strong negative prediction error), especially in subjects who obtained a better model fit. Additionally, also theta phase connectivity between midfrontal and task-related areas was increased after negative feedback. Thus, the data provided support for the hypothesis that the brain uses theta power and synchronization for flexibly switching between rules.

neuroscience↗

bioRxiv2018

Learning to synchronize: How biological agents can couple neural task modules for dealing with the stability-plasticity dilemma

We provide a novel computational framework on how biological and artificial agents can learn to flexibly couple and decouple neural task modules for cognitive processing. In this way, they can address the stability-plasticity dilemma. For this purpose, we combine two prominent computational neuroscience principles, namely Binding by Synchrony and Reinforcement Learning. The model learns to synchronize task-relevant modules, while also learning to desynchronize currently task-irrelevant modules. As a result, old (but currently task-irrelevant) information is protected from overwriting (stability) while new information can be learned quickly in currently task-relevant modules (plasticity). We combine learning to synchronize with several classical learning algorithms (backpropagation, Boltzmann machines, Rescorla-Wagner). For each case, we demonstrate that our combined model has significant computational advantages over the original network in both stability and plasticity. Importantly, the resulting models processing dynamics are also consistent with empirical data and provide empirically testable hypotheses for future MEG/EEG studies.\n\nAuthor summaryArtificial and biological agents alike face a critical trade-off between being sufficiently adaptive to acquiring novel information (plasticity) and retaining older information (stability); this is known as the stability-plasticity dilemma. Previous work on this dilemma has focused either on computationally efficient solutions for artificial agents or on biologically plausible frameworks for biological agents. What is lacking is a solution that combines computational efficiency with biological plausibility. Therefore, the current work proposes a computational framework on the stability-plasticity dilemma that provides empirically testable hypotheses on both neural and behavioral levels. In this framework, neural task modules can be flexibly coupled and decoupled depending on the task at hand. Testing this framework will allow us to gain more insight in how biological agents deal with the stability-plasticity dilemma.

neuroscience↗