Biology subjects

Schuck, N. W.

Publications and source records attributed to Schuck, N. W..

At least 19 recordsLinked to original sources

bioRxiv2025

Neural replay is connected to latent cause inference and supports fast generalization

Dynamic environments require generalizing knowledge through two computational mechanisms: Inferring clusters of equivalent observations and propagating reward expectations within a cluster. While neural replay has been linked to both, past work has not studied these processes simultaneously, leaving their relative importance for replay unclear. We used fMRI and computational modeling to investigate latent cause inference and replay in 52 participants during a value-based task with shared rewards across bandits. Participants learned this structure and achieved one-shot generalization after reversals, consistent with a latent cause inference process. fMRI during predecision pauses revealed backwards replay of shared-reward bandits in the visual cortex and medial temporal lobe. Trial-wise fluctuations in visual replay strength and content were better explained by value updates than by structure learning. Conversely, abstract reward structure representations were localized specifically to the MTL. These results demonstrate that on-task replay serves reward propagation within learned latent structures to facilitate fast generalization.

neuroscience↗

bioRxiv2025

Entorhinal cortex signals dimensions of past experience that can be generalised in a novel environment

No two situations are identical. They can be similar in some aspects but different in others. This poses a key challenge when attempting to generalise our experience from one situation to another. How do we distinguish the aspects that transfer across situations from those that do not? One hypothesis is that the entorhinal cortex (EC) meets this challenge by forming factorised representations that allow for increased neural similarity between events that share generalisable features. We tested this hypothesis using functional magnetic resonance imaging. Female and male participants (n=40) were trained to report behavioural sequences based on an underlying graph structure. Participants then made decisions in a new environment where some, but not all graph transitions from the previous structure could be generalised. Behavioural results showed that participants distinguished the generalisable transition information. Accuracy was significantly higher in blocks where sequence transitions were shared across environments, than those in which transitions differed. This boost in accuracy was especially pronounced during early exposure to the novel environment. Throughout this early phase, neural patterns in EC showed a corresponding differentiation of the generalisable aspects. Neural patterns representing starting locations in familiar and novel environments were significantly more similar in EC on trials where sequences could be generalised from prior experience, compared to trials with new sequential transitions. This signalling was associated with improved performance when prior sequence knowledge could be reused. Our results suggest that during early exposure to novel environments, EC may signal dimensions of past experience that can be generalised. Significance StatementGeneralisation is the process of using our experience to solve new challenges. A central problem we face when attempting to generalise is determining which aspects of a new situation are similar to what we have experienced before and which are different. This allows us to generalise our knowledge selectively, transferring insights to the aspects that are similar and relevant in novel scenarios, while avoiding over-generalisation to the aspects that differ. The results from this experiment suggest that the entorhinal cortex may distinguish these aspects during early exposure to new environments, supporting complex generalisation behaviour.

neuroscience↗

bioRxiv2025

Two time scales of adaptation in human learning rates

Different situations may require radically different information updating speeds (i.e., learning rates). Some demand fast learning rates, while others benefit from using slower ones. To adjust learning rates, decision makers could rely on either global, meta-learned differences between environments, or faster but transient adaptations to locally experienced prediction errors. Here, we introduce a new paradigm that allows researchers to measure and empirically disentangle both forms of adaptations. Participants performed short blocks of trials of a continuous estimation task - fishing for crabs - on six different islands that required different optimal (initial) learning rates. Across two experiments, participants showed fast adaptations in learning rate within a block. Critically, participants also learned global environment-specific learning rates over the time course of the experiment, as evidenced by computational modelling and by the learning rates calculated on the very first trial when revisiting an environment (i.e., unconfounded by transient adaptations). Using representational similarity analyses of fMRI data, we found that differences in voxel pattern responses in the central orbitofrontal cortex correlated with differences in these global environment-specific learning rates. Our findings show that humans adapt learning rates at both slow and fast time scales, and that the central orbitofrontal cortex may support meta-learning by representing environment-specific task-relevant features such as learning rates.

neuroscience↗

bioRxiv2025

Absence of Systematic Effects of Internalizing Psychopathology on Learning Under Uncertainty

Difficulties in adapting learning to meet the challenges of uncertain and changing environments are widely thought to play a central role in internalizing psychopathology, including anxiety and depression. This view stems from findings linking trait anxiety and transdiagnostic internalizing symptoms to learning impairments in laboratory tasks often used as proxies for real-world behavioral flexibility. These tasks typically require learners to adjust learning rates dynamically in response to uncertainty, for instance, increasing learning from prediction errors in volatile environments. However, prior studies have produced inconsistent and sometimes contradictory findings regarding the nature and extent of learning impairments in populations with internalizing disorders. To address this, we conducted eight experiments (N = 820) using predictive inference and reversal learning tasks, and applied a bi-factor analysis to capture internalizing symptom variance shared across and differentiated between anxiety and depression. While we observed robust evidence for adaptive learning-rate modulation across participants, we found no convincing evidence of a systematic relationship between internalizing symptoms and either learning rates or task performance. These findings challenge prominent claims that learning difficulties are a hallmark feature of internalizing psychopathology and suggest that the relationship between these traits and adaptive behavior under uncertainty may be more subtle than previously thought.

neuroscience↗

bioRxiv2024

N2 Sleep Inspires Insight

Humans sometimes have an insight that leads to a sudden and drastic performance im-provement on the task they are working on. The precise origins of such insights are unknown. Some evidence has shown that sleep facilitates insights, while other work has not found such a relationship. One recent suggestion that could explain this mixed evidence is that different sleep stages have differential effects on insight. In addition, computational work has suggested that neural variability and regularisation play a role in increasing the likelihood of insight. To investigate the link between insight and different sleep stages as well as regularisation, we conducted a preregistered study in which N=90 participants performed a perceptual insight task before and after a 20 minute daytime. Sleep EEG data showed that N2 sleep, but not N1 sleep, increases the likelihood of insight after a nap, suggesting a specific role of deeper sleep. Exploratory analyses of EEG power spectra showed that spectral slopes could predict insight beyond sleep stages, which is broadly in line with theoretical suggestions of a link between insight and regularisation. In combination, our findings point towards a role of N2 sleep and aperiodic, but not oscillatory, neural activity for insight.

neuroscience↗

bioRxiv2024

Neural Prioritisation of Past Solutions Supports Generalisation

Generalisation from past experience is an important feature of intelligent systems. When faced with a new task, one efficient computational approach is to evaluate solutions to earlier tasks as candidates for reuse. Consistent with this idea, we found that human participants (n=38) learned optimal solutions to a set of training tasks and generalised them to novel test tasks in a reward selective manner. This behaviour was consistent with a computational process based on the successor representation known as successor features and generalised policy improvement (SF&GPI). Neither model-free perseveration or model-based control using a complete model of the environment could explain choice behaviour. Decoding from functional magnetic resonance imaging data revealed that solutions from the SF&GPI algorithm were activated on test tasks in visual and prefrontal cortex. This activation had a functional connection to behaviour in that stronger activation of SF&GPI solutions in visual areas was associated with increased behavioural reuse. These findings point to a possible neural implementation of an adaptive algorithm for generalisation across tasks.

neuroscience↗

bioRxiv2024

Adaptive Integration of Perceptual and Reward Information in an Uncertain World

Perceptual uncertainty and salience both impact decision-making, but how these factors precisely impact trial-and-error reinforcement learning is not well understood. Here, we test the hypotheses that (H1) perceptual uncertainty modulates reward-based learning and that (H2) economic decision-making is driven by the value and the salience of sensory information. For this, we combined computational modeling with a perceptual uncertainty-augmented reward-learning task in a human behavioral experiment (N = 98). In line with our hypotheses, we found that subjects regulated learning behavior in response to the uncertainty with which they could distinguish choice options based on sensory information (belief state), in addition to the errors they made in predicting outcomes. Moreover, subjects considered a combination of expected values and sensory salience for economic decision-making. Taken together, this shows that perceptual and economic decision-making are closely intertwined and share a common basis for behavior in the real world.

neuroscience↗

bioRxiv2024

An inductive bias for slowly changing features in human reinforcement learning

Identifying goal-relevant features in novel environments is a central challenge for efficient behaviour. We asked whether humans address this challenge by relying on prior knowledge about common properties of reward-predicting features. One such property is the rate of change of features, given that behaviourally relevant processes tend to change on a slower timescale than noise. Hence, we asked whether humans are biased to learn more when task-relevant features are slow rather than fast. To test this idea, 295 human participants were asked to learn the rewards of two-dimensional bandits when either a slowly or quickly changing feature of the bandit predicted reward. Across two experiments and one preregistered replication, participants accrued more reward when a bandits relevant feature changed slowly, and its irrelevant feature quickly, as compared to the opposite. We did not find a difference in the ability to generalise to unseen feature values between conditions. Testing how feature speed could affect learning with a set of four function approximation Kalman filter models revealed that participants had a higher learning rate for the slow feature, and adjusted their learning to both the relevance and the speed of feature changes. The larger the improvement in participants performance for slow compared to fast bandits, the more strongly they adjusted their learning rates. These results provide evidence that human reinforcement learning favours slower features, suggesting a bias in how humans approach reward learning. Author summaryLearning experiments in the laboratory are often assumed to exist in a vacuum, where participants solve a given task independently of how they learn in more natural circumstances. But humans and other animals are in fact well known to "meta learn", i.e. to leverage generalisable assumptions about how to learn from other experiences. Taking inspiration from a well-known machine learning technique known as slow feature analysis, we investigated one specific instance of such an assumption in learning: the possibility that humans tend to focus on slowly rather than quickly changing features when learning about rewards. To test this, we developed a task where participants had to learn the value of stimuli composed of two features. Participants indeed learned better from a slowly rather than quickly changing feature that predicted reward. Computational modelling of participant behaviour indicated that participants had a higher learning rate for slowly changing features from the outset. Hence, our results support the idea that human reinforcement learning reflects a priori assumptions about the reward structure in natural environments.

neuroscience↗

bioRxiv2023

Grid-like entorhinal representation of an abstract value space during prospective decision making

Everyday decisions require us to predict how valuable different choice options will be in the future. Prior studies have identified a cognitive map in the hippocampal-entorhinal system that encodes relationships between states and enables prediction of future states, but does not inherently convey value during prospective decision making. Here, we investigated whether the entorhinal cortex integrates relational information about changing values by representing an abstract value space. To this end, we combined fMRI with a prospective decision making task that required participants to track and predict changing values of two choice options in a sequence. Such a sequence formed a trajectory through an underlying two-dimensional value space. Our results show that participants successfully integrated and extrapolated changes along the two value dimensions. Participants choice behavior was explained by a prospective reinforcement learning model and the degree to which they updated values over time correlated with self-reported navigational abilities and preferences. Crucially, while participants traversed the abstract value space, the entorhinal cortex exhibited a grid-like representation, with the phase of the hexadirectional fMRI signal (i.e., the orientation of the estimated grid) being aligned to the most informative axis through the value space. A network of brain regions, including the ventromedial prefrontal cortex (vmPFC), tracked the prospective value difference between options and the occipital-temporal cortex represented the more valuable option. These findings suggest that the entorhinal grid system might support the prediction of future values by representing a cognitive map, which might be used to generate lower-dimensional signals of the value difference between options and their identities for choices. Thus, these findings provide novel insight for our understanding of cognitive maps as a mechanism to guide prospective decision making in humans.

neuroscience↗

bioRxiv2023

Goal-seeking compresses neural codes for space in the human hippocampus and orbitofrontal cortex

Humans can navigate flexibly to meet their goals. Here, we asked how the neural representation of allocentric space is distorted by goal-directed behaviour. Participants navigated an agent to two successive goal locations in a grid world environment comprising four interlinked rooms, with a contextual cue indicating the conditional dependence of one goal location on another. Examining the neural geometry by which room and context were encoded in fMRI signals, we found that map-like representations of the environment emerged in both hippocampus and neocortex. Cognitive maps in hippocampus and orbitofrontal cortices were compressed so that locations cued as goals were coded together in neural state space, and these distortions predicted successful learning. This effect was captured by a computational model in which current and prospective locations are jointly encoded in a place code, providing a theory of how goals warp the neural representation of space in macroscopic neural signals.

neuroscience↗

bioRxiv2022

Trait anxiety is associated with hidden state inference during aversive reversal learning

Updating beliefs in changing environments can be driven by gradually adapting expectations or by relying on inferred hidden states (i.e., contexts), and changes therein. Previous work suggests that increased reliance on context could underly fear relapse phenomena that hinder clinical treatment of anxiety disorders. We test whether trait anxiety variations in a healthy population influence how much individuals rely on hidden-state inference. In a Pavlovian learning task, participants observed cues that predicted an upcoming electrical shock with repeatedly changing probability, and were asked to provide expectancy ratings on every trial. We show that trait anxiety is associated with steeper expectation switches after contingency reversals and reduced oddball learning. Furthermore, trait anxiety is related to better fit of a state inference compared to a gradual learning model when contingency changes are large. Our findings support previous work suggesting hidden-state inference as a mechanism behind anxiety-related to fear relapse phenomena.

neuroscience↗

bioRxiv2022

Statistical learning of successor representations is related to on-task replay

Humans automatically infer higher-order relationships between events in the environment from their statistical co-occurrence, often without conscious awareness. Neural replay of task representations is a candidate mechanism by which the brain learns such relational information or samples from a learned model in the service of adaptive behavior. Here, we tested whether cortical reactivation is related to learning higher-order sequential relationships without consciousness. Human participants viewed sequences of images that followed probabilistic transitions determined by ring-like graph structures. Behavioral modeling revealed that participants acquired multi-step transition knowledge through gradual updating of an internal successor representation (SR) model, although half of participants did not indicate conscious knowledge about the sequential task structure. To investigate neural replay, we analyzed the temporal dynamics of multivariate functional magnetic resonance imaging (fMRI) patterns during brief 10 seconds pauses from the ongoing statistical learning task. We found evidence for backward sequential replay of multi-step sequences in visual cortical areas. These findings indicate that implicit learning of higher-order relationships establishes an internal SR-based map of the task, and is accompanied by cortical on-task replay.

neuroscience↗

bioRxiv2021

Hippocampal spatio-temporal cognitive maps adaptively guide reward generalization

The brain forms cognitive maps of relational knowledge, an organizing principle thought to underlie our ability to generalize and make inferences. However, how can a relevant map be selected in situations where a stimulus is embedded in multiple relational structures? Here, we find that both spatial and temporal cognitive maps influence generalization in a choice task, where spatial location determines reward magnitude. Mirroring behavior, the hippocampus not only builds a map of spatial relationships but also encodes temporal distances. As the task progresses, participants choices become more influenced by spatial relationships, reflected in a strengthening of the spatial and a weakening of the temporal map. This change is driven by orbitofrontal cortex, which represents the evidence that an observed outcome is generated from the spatial rather than the temporal map and updates hippocampal representations accordingly. Taken together, this demonstrates how hippocampal cognitive maps are used and updated flexibly for inference.

neuroscience↗

bioRxiv2021

L-DOPA enhances hippocampal directionsignals in younger and older adults

Previous studies indicate a role of dopamine in hippocampus-dependent spatial navigation. Although neural representations of direction are an important aspect of spatial cognition, it is not well understood whether dopamine directly affects these representations, or only impacts other aspects of spatial brain function. Moreover, both dopamine and spatial cognition decline sharply during age, raising the question which effect dopamine has on directional signals in the brain of older adults. To investigate these questions, we used a double-blind cross-over L-DOPA/Placebo intervention design in which 43 younger and 37 older adults navigated in a virtual spatial environment while undergoing functional magnetic resonance imaging (fMRI). We studied the effect of L-DOPA, a DA precursor, on fMRI activation patterns that encode spatial walking directions that have previously been shown to lose specificity with age. This was done in predefined regions of interest, including the early visual cortex, retrosplenial cortex, and hippocampus. Classification of brain activation patterns associated with different walking directions was improved in the hippocampus and the retrosplenial cortex following L-DOPA administration. This suggests that DA enhances the specificity of neural representations of walking direction in these areas. In the hippocampus these results were found in both age groups, while in the RSC they were only observed in younger adults. Taken together, our study provides evidence for a mechanistic link between DA and the specificity of neural responses during spatial navigation. Significance StatementThe sense of direction is an important aspect of spatial navigation, and neural representations of direction can be found throughout a large network of space-related brain regions. But what influences how well these representations track someones true direction? Using a double-blind cross-over L-DOPA/Placebo intervention design, we find causal evidence that the neurotransmitter dopamine impacts the fidelity of direction selective neural representations in the human hippocampus and retrosplenial cortex. Interestingly, the effect of L-DOPA was either equally present or even smaller in older adults, despite the well-known age related decline of dopamine. These results provide novel insights into how dopamine shapes the neural representations that underlie spatial navigation.

neuroscience↗

bioRxiv2021

Parallel representation of context and multiple context-dependent values in ventro-medial prefrontal cortex

Value representations in ventromedial prefrontal-cortex (vmPFC) are known to guide the choice between options. But the value of an option can be different in different task contexts. Goal-directed behavior therefore requires to know the current context and associated values of options, and to flexibly switch between value representations in a task-dependent manner. We tested whether task-relevant and -irrelevant values influence behavior and asked whether both values are represented together with context signals in vmPFC. Thirty-five participants alternated between tasks in which stimulus color or motion predicted rewards. As expected, neural activity in vmPFC and choices were largely driven by task-relevant values. Yet, behavioral and neural analyses indicate that participants also retrieved the values of irrelevant features, and computed which option would have been best in the alternative context. Investigating the probability distributions over values and contexts encoded in multivariate fMRI signals, we find that vmPFC maintains representations of the current context, i.e. task state, the value associated with it, and the hypothetical value of the alternative task state. Crucially, we show that evidence for irrelevant value signals in vmPFC relates to behavior on multiple levels, competes with expected value signals, and interacts with task state representations. Our results thus suggest that different value representations are represented in parallel and imply a link between neural representations of task states, their associated values and their influence on behavior. This sheds new light on vmPFCs role in decision making, bridging between a hypothesized role in mapping observations onto the task states of a mental map, and computing value expectations for alternative states.

neuroscience↗

bioRxiv2021

Control over patch encounters changes foraging behaviour

Foraging is a common decision problem in natural environments. When new exploitable sites are always available, a simple optimal strategy is to leave a current site when its return falls below a single average reward rate. Here, we examined foraging in a more structured environment, with a limited number of sites that replenished at different rates and had to be revisited. When participants could choose sites, they visited fast-replenishing sites more often, left sites at higher levels of reward, and achieved a higher net reward rate. Decisions to exploit-or-leave a site were best explained with a computational model that included both the average reward rate for the environment and reward information about the unattended sites. This suggests that unattended sites influence leave decisions, in foraging environments where sites can be revisited. HighlightsO_LIBeing able to select sites during foraging increased visits to high value sites C_LIO_LIThis visitation pattern was efficient, producing higher average reward rates C_LIO_LIDecisions to leave a site were influenced by information about alternative sites C_LI Graphical Abstract O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=200 SRC="FIGDIR/small/426950v3_ufig1.gif" ALT="Figure 1"> View larger version (37K): org.highwire.dtl.DTLVardef@2e662org.highwire.dtl.DTLVardef@6352eaorg.highwire.dtl.DTLVardef@d8aae7org.highwire.dtl.DTLVardef@bc60a6_HPS_FORMAT_FIGEXP M_FIG C_FIG

animal behavior and cognition↗

bioRxiv2020

Orbitofrontal cortex and learning predictions of state transitions

Learning the transition structure of the environment - the probabilities of transitioning from one environmental state to another - is a key prerequisite for goal-directed planning and model-based decision making. To investigate the role of the orbitofrontal cortex (OFC) in goal-directed planning and decision making, we used fMRI to assess univariate and multivariate activity in the OFC while humans experienced state transitions that varied in degree of surprise. Converging with recent evidence, we found that OFC activity was related to greater learning about transition structure. However, the observed relationship was inconsistent with a straightforward interpretation of OFC activity as representing a state prediction error that would facilitate learning of transitions via error-correcting mechanisms. The state prediction error hypothesis predicts that OFC activity at the time of observing an outcome should increase expectation of that same observed outcome on subsequent trials. Instead, our results showed that OFC activity was associated with increased expectation of the more probable outcome; that is, with more optimal predictions. These results suggest that the OFC is involved in updating or reinforcing a learned transition model on a trial-by-trial basis, specifically for the currently observed cue-outcome associations. Our findings add to the evidence of OFC involvement in learning state-to-state transition structure, while providing new constraints for algorithmic hypotheses regarding how these transitions are learned. Significance StatementThe orbitofrontal cortex (OFC) has been implicated in model-based decision making--the kind of decisions that result from planning using an "environment model" of how current actions affect our future states. However, the widely suggested role of the OFC in representing expected values of future states is not sufficient to explain why the OFC would be critical for planning in particular. A new line of evidence implicates the OFC in learning about transition structure of the environment - a key component of the "environment model" used for planning. We investigate this function, adding to the growing literature on the role of the OFC in learning and decision making, while unveiling new questions about the algorithmic role of OFC in goal-directed planning.

neuroscience↗

bioRxiv2020

Faster than thought: Detecting sub-second activation sequences with sequential fMRI pattern analysis

Neural computations are often anatomically localized and executed on sub-second time scales. Understanding the brain therefore requires methods that offer sufficient spatial and temporal resolution. This poses a particular challenge for the study of the human brain because non-invasive methods have either high temporal or spatial resolution, but not both. Here, we introduce a novel multivariate analysis method for conventional blood-oxygen-level dependent functional magnetic resonance imaging (BOLD fMRI) that allows to study sequentially activated neural patterns separated by less than 100 ms with anatomical precision. Human participants underwent fMRI and were presented with sequences of visual stimuli separated by 32 to 2048 ms. Probabilistic pattern classifiers were trained on fMRI data to detect the presence of image-specific activation patterns in early visual and ventral temporal cortex. The classifiers were then applied to data recorded during sequences of the same images presented at increasing speeds. Our results show that probabilistic classifier time courses allowed to detect neural representations and their order, even when images were separated by only 32 ms. Moreover, the frequency spectrum of the statistical sequentiality metric distinguished between sequence speeds on sub-second versus supra-second time scales. These results survived when data with high levels of noise and rare sequence events at unknown times were analyzed. Our method promises to lay the groundwork for novel investigations of fast neural computations in the human brain, such as hippocampal replay.

neuroscience↗