bioRxiv Science⌕ Search

Biology subjects

Kikumoto, A.

Publications and source records attributed to Kikumoto, A..

11 recordsLinked to original sources

Goal-directed action selection relies on conjunctive representations that bind goals, actions and expected sensory outcomes

Goal-directed actions are performed to obtain specific sensory outcomes. However, theories of action control diverge on whether anticipated outcomes are only consequences of motor command specification or serve as instrumental determinants of action selection. Here, we directly tested whether predicted sensory outcomes are integrated into the neural processes underlying the selection of goal-directed actions. Participants learned novel action-outcome contingencies and selected actions to generate specific sensory outcomes, while their brain activity was recorded using EEG. Combining time-resolved multivariate decoding with representational similarity analysis, we tracked both feature-specific and conjunctive neural representations over time. We found that the strength of high-dimensional conjunctive representations including anticipated sensory outcomes predicted trial-to-trial reaction times, even after accounting for individual task features. These results provide neural evidence that predicted sensory outcomes are not downstream consequences, but core determinants of action selection, supporting the view that actions are selected to control expected sensory input.

neuroscience↗

The unique value of zero prediction errors in reinforcement learning

Updating beliefs when necessary is at the cornerstone of learning. A fundamental problem is to describe under which conditions humans update their internal models of the world. The general assumption across animal, human, and artificial learning models1-3 has been that updates occur when outcomes deviate from expectations. Perfectly predicted outcomes cause no learning. As a result, no research has examined cases in which predictions are surprisingly perfect. Here, we empirically test this assumption and find that zero prediction errors are psychologically unique. We show that after zero prediction errors, momentary happiness is the highest, and belief updates do indeed occur in a pattern that cannot be reproduced by a benchmark model. We present a new model that captures this non-linear pattern in belief updating by postulating that zero prediction errors elicit a distinct latent belief state, guiding subsequent updating. This latent state then tracks neural activity patterns measured with EEG precisely when zero prediction errors occur, exactly as the model would predict. Crucially, the strength of the neural activity during this time window exhibits a dissociation in predicting the next belief update depending on whether feedback was a zero prediction error or a regular prediction error. Overall, we provide strong evidence that surprisingly perfect predictions are treated in a unique, non-linear fashion at affective, behavioral and neural levels. Being surprisingly accurate can function as a distinct belief updating signal, conforming trial-and-hit learning.

neuroscience↗

Neural and behavioral signatures of policy compression in cognitive control

Making context-dependent decisions incurs cognitive costs. Cognitive control studies have investigated the nature of such costs from both computational and neural perspectives. In this paper, we offer an information-theoretic account of the costs associated with context-dependent decisions. According to this account, the brains limited capacity to store context-dependent policies necessitates "compression" of policies into internal representations with an upper bound on code length, quantified by an information-theoretic measure (policy complexity). These representations are decoded into actions by sequentially inspecting each bit, such that longer codes take more time to decode. When a response deadline is imposed, the account predicts that policy complexity should increase with the deadline. Higher policy complexity is associated with several behavioral signatures: (i) higher accuracy; (ii) lower variability; and (iii) lower perseveration. Analyzing data from a rule-based action selection task, we found evidence supporting all of these predictions. We further hypothesized that complex policies require higher neural dimensionality (which constrains the code space). Consistent with this hypothesis, we found that policy complexity correlates with a measure of neural dimensionality in a rule-based decision task. This finding brings us a step closer to understanding the neural implementation of policy compression and its implications for cognitive control.

neuroscience↗

Practice Reshapes the Geometry and Dynamics of Task-tailored Representations

Extensive practice makes task performance more efficient and precise, leading to automaticity. However, theories of automaticity differ on which levels of task representations (e.g., low-level features, stimulus-response mappings, or high-level conjunctive memories of individual events) change with practice, despite predicting the same pattern of improvement (e.g., power law of practice). To resolve this controversy, we built on recent theoretical advances in understanding computations through neural population dynamics. Specifically, we hypothesized that practice optimizes the neural representational geometry of task representations to minimally separate the highest-level task contingencies needed for successful performance. This involves efficiently reaching conjunctive neural states that integrate task-critical features nonlinearly while abstracting over non-critical dimensions. To test this hypothesis, human participants (n = 40) engaged in extensive practice of a simple, context-dependent action selection task over 3 days while recording EEG. During initial rapid improvement in task performance, representations of the highest-level, context-specific conjunctions of task-features were enhanced as a function of the number of successful episodes. Crucially, only enhancement of these conjunctive representations, and not lower-order representations, predicted the power-law improvement in performance. Simultaneously, over sessions, these conjunctive neural states became more stable earlier in time and more aligned, abstracting over redundant task features, which correlated with offline performance gain in reducing switch costs. Thus, practice optimizes the dynamic representational geometry as task-tailored neural states that minimally tesselate the task space, taming their high-dimensionality.

neuroscience↗

A Transient High-dimensional Geometry Affords Stable Conjunctive Subspaces for Efficient Action Selection

Flexible action selection requires cognitive control mechanisms capable of mapping the same inputs to different output actions depending on the context. From a neural state-space perspective, this requires a control representation that separates similar input neural states by context. Additionally, for action selection to be robust and time-invariant, information must be stable in time, enabling efficient readout. Here, using EEG decoding methods, we investigate how the geometry and dynamics of control representations constrain flexible action selection in the human brain. Participants performed a context-dependent action selection task. A forced response procedure probed action selection different states in neural trajectories. The result shows that before successful responses, there is a transient expansion of representational dimensionality that separated conjunctive subspaces. Further, the dynamics stabilizes in the same time window, with entry into this stable, high-dimensional state predictive of individual trial performance. These results establish the neural geometry and dynamics the human brain needs for flexible control over behavior.

neuroscience↗

The Role of Conjunctive Representations in Prioritizing and Selecting Planned Actions

For flexible goal-directed behavior, prioritizing and selecting a specific action among multiple candidates is often important. Working memory has long been assumed to play a role in prioritization and planning, while bridging cross-temporal contingencies during action selection. However, studies of working memory have mostly focused on memory for single components of an action plan, such as a rule or a stimulus, rather than management of all of these elements during planning. Therefore, it is not known how post-encoding prioritization and selection operate on the entire profile of representations for prospective actions. Here, we assessed how such control processes unfold over action representations, highlighting the role of conjunctive representations that nonlinearly integrate task-relevant features during maintenance and prioritization of action plans. For each trial, participants prepared two independent rule-based actions simultaneously, then they were retro-cued to select one as their response. Prior to the start of the trial, one rule-based action was randomly assigned to be high priority by cueing that it was more likely to be tested. We found that both full action plans were maintained as conjunctive representations during action preparation, regardless of priority. However, during output selection, the conjunctive representation of the high priority action plan was more enhanced and readily selected as an output. Further, the strength of conjunctive representation was related to behavioral interference when the low priority action was tested. Thus, multiple integrated representations were maintained for upcoming actions and served as the target of post-encoding attentional selection mechanisms to prioritize and select an action from those in working memory.

neuroscience↗

The Role of Conjunctive Representations in Controlling Actions

Action selection appears to rely on conjunctive representations that nonlinearly integrate task-relevant features (Kikumoto & Mayr, 2020). We test here the corollary hypothesis that such representations are also intricately involved during attempts to stop an action--a key aspect of action regulation. We tracked both conjunctive representations and those of constituent rule, stimulus, or response features through trial-by-trial representational similarity analysis of the EEG signal in a combined, rule-selection and stop-signal paradigm. Across two experiments with student participants (N = 57), we found (a) that the strength of decoded conjunctive representations prior to the stop signal uniquely predicted trial-by-trial stopping success (Exp. 1) and (b) that these representations were selectively suppressed following the onset of the stop signal (Exp. 1 and 2). We conclude that conjunctive representations are key to successful action execution and therefore need to be suppressed when an intended action is no longer appropriate. Statement of RelevanceSome theorists have posited that as a necessary step during action selection, action-relevant features need to be combined within a conjunctive representation that is more than the sum if its basic features. Consequently, such representations should also play a critical role when trying to stop an intended action--a key aspect of self-regulation. However direct evidence of conjunctive representations has been elusive. Using a method for tracking both conjunctive and basic-feature representations on a trial-by-trial basis in the EEG signal, we show that the stronger the conjunctive representations, the harder it was to stop the intended action. Furthermore, the stopping process also selectively reduced the strength of conjunctive representations. These results further our knowledge about action regulation by showing that conjunctive representations are a necessary precursor for carrying out actions successfully and for that reason also need to be the target of self-regulatory stopping attempts.

neuroscience↗

Conjunctive Representations that Integrate Stimuli, Responses, and Rules are Critical for Action Selection

People can use abstract rules to flexibly configure and select actions for specific situations. Yet how exactly rules shape actions towards specific sensory and/or motor requirements remains unclear. One possibility is that rules become integrated with sensory/response features in a non-linear, conjunctive manner (e.g., event files; Hommel, 1998) to drive rule-guided action selection. To dynamically track such conjunctive representations during action selection, we applied a time-resolved representational similarity analysis to the spectral-temporal profiles of the EEG signal, while participants selected actions based on varying rules. Across two experiments, we found that action selection engages conjunctive representations binding action rules to specific sensory/motor settings throughout the entire selection period. The strength of conjunctions was the most important predictor of trial-by-trial variability in response times (RTs) and was closely, and selectively, related to an important behavioral indicator of event files--the partial-overlap priming pattern. Thus, conjunctive representations were functionally dissociated from their constituent action features and play a critical role during flexible selection of action.

neuroscience↗

Balancing Model-Based and Memory-Free Action Selection under Competitive Pressure

In competitive situations, winning depends on selecting actions that surprise the opponent. Such unpredictable action can be generated based on representations of the opponents strategy and choice history (model-based counter-prediction) or by choosing actions in a memory-free, stochastic manner. Across five different experiments using a variant of a matching-pennies game with simulated and human opponents we found that people toggle between these two strategies, using model-based selection used when recent wins signal the appropriateness of the current model, but reverting to stochastic selection following losses. Also, after wins, feedback-related, mid-frontal EEG activity reflected information about the opponents global and local strategy, and predicted upcoming choices. After losses, this activity was nearly absent--indicating that the internal model is suppressed after negative feedback. We suggest that the mixed-strategy approach allows negotiating two conflicting goals: (1) exploiting the opponents deviations from randomness while (2) remaining unpredictable for the opponent.

neuroscience↗

Decoding Hierarchical Control of Sequential Behavior in Oscillatory EEG Activity

Despite strong theoretical reasons for assuming that abstract representations organize complex action sequences in terms of subplans (chunks) and sequential positions, we lack methods to directly track such content-independent, hierarchical representations in humans. We applied time-resolved, multivariate decoding analysis to the pattern of rhythmic EEG activity that was registered while participants planned and executed individual elements from pre-learned, structured sequences. Across three experiments, the theta and alpha-band activity independently coded basic elements and abstract control representations, in particular the ordinal position of basic elements, but also the identity and position of chunks. Further, a robust representation of higher-level, chunk identity information was only found in individuals with above-median working memory capacity, potentially providing a neural-level explanation for working-memory differences in sequential performance. Our results suggest that by decoding oscillations we can track how the cognitive system traverses through the states of a hierarchical control structure.

neuroscience↗

EEG Decoding Reveals the Temporal Dynamics and Functional Relevance of Goal-Relevant Representations

Models of action control assume that abstract task-set settings regulate lower-level stimulus/response representations. Yet, we know little about the functional and dynamic properties of task-set representations in humans. Using a cued task-switching paradigm, we show that information about task sets and lower-level stimulus/response aspects can be extracted through decoding analyses from the scalp electrophysiological signal (EEG) on the single-trial level and with high temporal resolution. Task-sets are active throughout the entire processing cascade and trial-to-trial variations in task-set strength emerges as a remarkably strong predictor of variability in performance, both within and between individuals. Also, taskset strength is related to stimulus representation strength at an early period and to the strength of response representations at a later period, consistent with the notion that task-sets coordinate successive, lower-level representations in a concurrent manner. These results demonstrate a powerful approach towards uncovering stages of information processing and their relative importance for performance.

neuroscience↗