Biology subjects

Akam, T.

Publications and source records attributed to Akam, T..

17 recordsLinked to original sources

bioRxiv2026

Transition from model-free to structure-informed decision making in dynamic environments

Reinforcement learning theory formulates distinct decision-making strategies, including reactive model-free and deliberative model-based strategies. This study investigates how mice adjust their reinforcement learning strategies while learning decision-making in dynamic environments. Unlike previous studies that focused on behaviors after extensive training periods, we analyzed changes in learning strategies in the course of training of a two-step decision-making task with probabilistic state transition and fluctuating reward probabilities. Our statistical behavioral analysis showed that the stay-probability following common and rare transitions diverged with training, a signature of strategies that utilize knowledge of task structure. We fit various reinforcement learning strategies to behavioral data and found that structure-informed strategies became increasingly dominant in their behaviors during training. Whereas previous studies emphasized transition from goal-directed to habitual strategies after extensive training, which were often associated with model-based and model-free strategies, respectively, our results newly demonstrate a shift from model-free to structure-informed strategies in early training in mice. Author summaryReinforcement learning theory allows us to examine how we make decisions and what approaches we use to optimize rewards. Most previous research, however, has examined animal behavior only after extensive training. Here we analyzed how mice adjust their reinforcement learning strategies as they are trained in a two-step decision-making task. Initially, mice relied on reactive model-free strategies, but as training progressed, their behavior began to incorporate knowledge of task structure. While previous studies suggested transition from model-based to model-free strategies with extensive training, our study revealed the opposite in the early stage of training.

animal behavior and cognition↗

bioRxiv2026

Learning complex temporal dependencies via local synaptic plasticity

The ability to extract and exploit temporal structure across diverse tasks is central to human cognition. Neuroscientists have typically relied on recurrent neural networks (RNNs) trained with backpropagation through time (BPTT) when modelling neural and behavioural processes such as decision-making and motor control. However, this algorithm has limited biological plausibility, hence the computational principles underlying efficient learning of temporal dependencies remain unresolved. Here, we investigate temporal predictive coding (tPC), a recently proposed framework that extends predictive coding to the temporal domain while preserving local Hebbian update rules. We analyse and extend tPC to establish its relationship with several influential computational models of learning in RNNs, including BPTT, reservoir computing, and eligibility propagation (e-prop). We first demonstrate a functional equivalence between tPC and tBPTT1, a variant of BPTT in which gradients are propagated only one time step into the past. We then show that tPC can leverage reservoir dynamics to encode short-range temporal context, and simultaneously sculpt neural trajectories in state space to support downstream readout. We further demonstrate that hierarchical recurrent dynamics can facilitate learning of more complex temporal dependencies, while additionally conferring robustness to strong distractors. Finally, we show that tPC networks can be augmented with biologically inspired eligibility traces to solve temporally extended context-dependent tasks. Together, these results reveal that relatively simple recurrent networks governed by local plasticity can support temporal learning in more complex settings than previously appreciated. Author summaryTo navigate the world, our brains must constantly track how events unfold over time, whether we are predicting the next word in a sentence or timing a tennis swing. Neuroscientists often use artificial neural networks to study how the brain learns these sequences. However, these models are often not biologically realistic, since they adjust neuronal connections using information that individual neurons would not have access to in the brain. Here, we examine an alternative framework called temporal predictive coding, which is designed to better reflect how brain networks may be organised. We show that these biologically inspired networks are surprisingly powerful: they can remember recent context and, when arranged in layers, can learn increasingly complex patterns. We also show that they work better when equipped with a memory-like mechanism that helps neurons link recent events across time. Overall, our findings suggest that relatively simple, biologically inspired networks may capture how we process sequential events better than previously thought.

neuroscience↗

bioRxiv2026

Flexible route planning and rapid structure learning by mice in complex environments

Action selection using predictive models of the environment plays a fundamental role in human and animal behaviour, yet is poorly understood at circuit and algorithmic levels. Spatial navigation is an attractive domain for characterising how world models guide action selection. However spatial behaviours are shaped by multiple control systems including habits, vector-navigation using a Euclidean model of spatial relationships, and route planning using models of environment structure. Understanding how world models support navigation requires assays that dissociate control systems and decorrelate behavioural variables, while generating large datasets that allow precise quantification of brain-behaviour relationships. Here we developed and computationally optimised a behavioural assay to quantify flexible navigation using knowledge of environment structure. Mice navigated to visually cued goals in complex mazes, with randomised start and goal locations on each trial, generating thousands of non-repetitive goal-directed navigation trajectories. They navigated efficiently - strongly favouring options on the shortest path-to-goal, and learnt rapidly - demonstrating knowledge of maze structure from their first sessions in new environments. We anticipate the assay will be useful for characterising how world models support flexible behaviour.

animal behavior and cognition↗

bioRxiv2026

Dopamine ramps as a normative consequence of dual-process control

1Midbrain dopamine neurons are thought to implement a temporal difference (TD) reward prediction error (RPE) that updates cached values stored in striatum. This has been challenged by evidence that dopamine "ramps up" to predictable rewards during goal-directed behaviour. Here, we propose that dopamine ramps are RPEs generated by a dual-process learning system in which values inferred using a world model train cached values via the RPE. Ramps arise because efficient training of cached values requires that inferred values contribute to the update target but not the prediction component of the RPE. The model reproduces key dopamine ramp phenomena, including learning dynamics on fast and slow timescales, global updates following changes in reward expectation, transient responses during unexpected state transitions, and sensitivity to state uncertainty manipulations. We therefore argue that dopamine ramps are a signature of interactions between inferred and cached values that revise the traditional dichotomy between model-based and model-free learning.

neuroscience↗

bioRxiv2025

A mechanistic theory of planning in prefrontal cortex

Planning is critical for adaptive behaviour in a changing world, because it lets us anticipate the future and adjust our actions accordingly. While prefrontal cortex is crucial for this process, it remains unknown how planning is implemented in neural circuits. Prefrontal representations were recently discovered in simpler sequence memory tasks, where different populations of neurons represent different future time points. We demonstrate that combining such representations with the ubiquitous principle of neural attractor dynamics allows circuits to solve much richer problems including planning. This is achieved by embedding the environment structure directly in synaptic connections to implement an attractor network that infers desirable futures. The resulting 'spacetime attractor' excels at planning in challenging tasks known to depend on prefrontal cortex. Recurrent neural networks trained by gradient descent on such tasks learn a solution that precisely recapitulates the spacetime attractor - in representation, in dynamics, and in connectivity. Analyses of networks trained across different environment structures reveal a generalisation mechanism that rapidly reconfigures the world model used for planning, without the need for synaptic plasticity. The spacetime attractor is a testable mechanistic theory of planning. If true, it would provide a path towards detailed mechanistic understanding of how prefrontal cortex structures adaptive behaviour.

neuroscience↗

bioRxiv2024

Excitatory neurons of the anterior cingulate cortex encode chosen actions and their outcomes rather than cognitive state

The anterior cingulate cortex (ACC) causally influences cognitive control of goal-directed behaviour. However, it is unclear whether ACC directly encodes cognitive variables like attention or impulsivity, or implements goal-directed action selection mechanisms that are modulated by them. We recorded ACC activity with miniature endoscopic microscopes in mice performing the 5-choice-serial-reaction time task, and applied decoding and encoding analyses. ACC pyramidal cells represented specific actions before and during the behavioural response, whereas the response type (e.g. correct/incorrect/premature) - indicating the state of attentional and impulse control - could only be decoded during and after the response with high reliability. Devaluation and extinction experiments further revealed that action encoding depended on reward expectation. Our findings support a role for ACC in goal-directed action selection and monitoring, that is modulated by cognitive state, rather than in tracking levels of attention or impulsivity directly in individual trials.

neuroscience↗

bioRxiv2024

Prioritizing replay when future goals are unknown

Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals. These results may support an alternative hypothesis that replay updates route information to build a "cognitive map." Yet no similar theory exists to formalize this view, it is unclear how such a map is represented or what role replay plays in computing it. We address these gaps by introducing a theory of replay that learns a map of routes to candidate goals, before reward is available or when its location may change. Replay is then focused on current goals (as with planning) and/or potential future goals (like a map), depending on the animals expectations about future goal switching. Our work thus generalizes the planning account to capture a general map-building function for replay, reconciling it with data, and revealing an unexpected relationship between the seemingly distinct hypotheses. The theory offers a unifying explanation why data from tasks with different goal dynamics have seemingly supported different hypotheses for the function of replay, and suggests new predictions for experiments testing these effects. Graphical Abstract HighlightsO_LIA reinforcement learning model explains how to use replay to build a cognitive map for navigation C_LIO_LIMap-building replay explains recent replay data from goal-switching tasks that stymie previous theoretical accounts C_LIO_LIModel predicts replay in the absence of rewards as well as sensitivity of replay to goal statistics C_LIO_LIModel suggests planning-like replay is a special case of map-like replay C_LI

neuroscience↗

bioRxiv2023

A Cellular Basis for Mapping Behavioural Structure

To flexibly adapt to new situations, our brains must understand the regularities in the world, but also in our own patterns of behaviour. A wealth of findings is beginning to reveal the algorithms we use to map the outside world1-6. In contrast, the biological algorithms that map the complex structured behaviours we compose to reach our goals remain enigmatic. Here we reveal a neuronal implementation of an algorithm for mapping abstract behavioural structure and transferring it to new scenarios. We trained mice on many tasks which shared a common structure organising a sequence of goals, but differed in the specific goal locations. Animals discovered the underlying task structure, enabling zero-shot inferences on the first trial of new tasks. The activity of most neurons in the medial Frontal cortex tiled progress-to-goal, akin to how place cells map physical space. These "goal-progress cells" generalised, stretching and compressing their tiling to accommodate different goal distances. In contrast, progress along the overall sequence of goals was not encoded explicitly. Instead a subset of goal-progress cells was further tuned such that individual neurons fired with a fixed task-lag from a particular behavioural step. Together these cells implemented an algorithm that instantaneously encoded the entire sequence of future behavioural steps, and whose dynamics automatically retrieved the appropriate action at each step. These dynamics mirrored the abstract task structure both on-task and during offline sleep. Our findings suggest that goal-progress cells in the medial frontal cortex may be elemental building blocks of schemata that can be sculpted to represent complex behavioural structures.

neuroscience↗

bioRxiv2021

Perception and propagation of activity through the cortical hierarchy is determined by neural variability

The brains of higher organisms are composed of anatomically and functionally distinct regions performing specialised tasks; but regions do not operate in isolation. Orchestration of complex behaviours requires communication between brain regions, but how neural activity dynamics are organised to facilitate reliable transmission is not well understood. We studied this process directly by generating neural activity that propagates between brain regions and drives behaviour, allowing us to assess how populations of neurons in sensory cortex cooperate to transmit information. We achieved this by imaging two hierarchically organised and densely interconnected regions, the primary and secondary somatosensory cortex (S1 and S2) in mice while performing two-photon photostimulation of S1 neurons and assigning behavioural salience to the photostimulation. We found that the probability of perception is determined not only by the strength of the photostimulation signal, but also by the variability of S1 neural activity. Therefore, maximising the signal-to-noise ratio of the stimulus representation in cortex relative to the noise or variability in cortex is critical to facilitate activity propagation and perception. Further, we show that propagated, behaviourally salient activity elicits balanced, persistent, and generalised activation of the downstream region. Hence, our work adds to existing understanding of cortical function by identifying how population activity is formatted to ensure robust transmission of information, allowing specialised brain regions to communicate and coordinate behaviour.

neuroscience↗

bioRxiv2021

Retrosplenial cortex is necessary for spatial and non-spatial latent learning in mice

Latent learning occurs when associations are formed between stimuli in the absence of explicit reinforcement. Traditionally, latent learning in rodents has been associated with the creation internal models of space. However, increasing evidence points to roles of internal models also in non-spatial decision making. Whether the same brain structures and processes support the creation of spatially-anchored or non-spatial internal models via latent learning, is an open question. To address this question, we developed a novel operant box task that allows to test spatial and non-spatial versions of a flavour-based sensory preconditioning paradigm. We probed the role of the retrosplenial cortex, a brain area associated with spatial cognition and subjective value representation, in this task using precise, closed-loop optogenetic silencing during different task phases. We show that the retrosplenial cortex is necessary for both spatial and non-spatial latent learning in mice. We further demonstrate that the requirement of retrosplenial cortex is limited to the preconditioning phase of the task. Our results provide insight into the specific role of the retrosplenial cortex in latent learning, demonstrate that latent learning plays a general part in the creation of internal models, independent of spatial anchors, and provide a novel avenue for studying model-based decision making.

neuroscience↗

bioRxiv2021

Dopamine reports reward prediction errors, but does not update policy, during inference-guided choice

Rewards are thought to influence future choices through dopaminergic reward prediction errors (RPEs) updating stored value estimates. However, accumulating evidence suggests that inference about hidden states of the environment may underlie much adaptive behaviour, and it is unclear how these two accounts of reward-guided decision-making should be integrated. Using a two-step task for mice, we show that dopamine reports RPEs using value information inferred from task structure knowledge, alongside information about recent reward rate and movement. Nonetheless, although rewards strongly influenced choices and dopamine, neither activating nor inhibiting dopamine neurons at trial outcome affected future choice. These data were recapitulated by a neural network model in which frontal cortex learned to track hidden task states by predicting observations, while basal ganglia learned corresponding values and actions via dopaminergic RPEs. Together, this two-process account reconciles how dopamine-independent state inference and dopamine-mediated reinforcement learning interact on different timescales to determine reward-guided choices.

neuroscience↗

bioRxiv2021

Highly task-specific and distributed neural connectivity in working memory revealed by single-trial decoding in mice and humans

Working memory (WM), the capacity to briefly and intentionally maintain mental items, is key to successful goal-directed behaviour and impaired in a range of psychiatric disorders. To date, several brain regions, connections, and types of neural activity have been correlatively associated with WM performance. However, no unifying framework to integrate these findings exits, as the degree of their species- and task-specificity remains unclear. Here, we investigate WM correlates in three task paradigms each in mice and humans, with simultaneous multi-site electrophysiological recordings. We developed a machine learning-based approach to decode WM-mediated choices in individual trials across subjects from hundreds of electrophysiological measures of neural connectivity with up to 90% prediction accuracy. Relying on predictive power as indicator of correlates of psychological functions, we unveiled a large number of task phase-specific WM-related connectivity from analysis of predictor weights in an unbiased manner. Only a few common connectivity patterns emerged across tasks. In rodents, these were thalamus-prefrontal cortex delta- and beta-frequency connectivity during memory encoding and maintenance, respectively, and hippocampal-prefrontal delta- and theta-range coupling during retrieval, in rodents. In humans, task-independent WM correlates were exclusively in the gamma-band. Mostly, however, the predictive activity patterns were unexpectedly specific to each task and always widely distributed across brain regions. Our results suggest that individual tasks cannot be used to uncover generic physiological correlates of the psychological construct termed WM and call for a new conceptualization of this cognitive domain in translational psychiatry.

neuroscience↗

bioRxiv2021

Complementary task representationsin hippocampus and prefrontal cortexfor generalising the structure of problems

Few situations in life are completely novel. We effortlessly generalise prior knowledge to solve novel problems, abstracting common structure and mapping it onto new sensorimotor specifics. Here we trained mice on a series of reversal learning problems that shared the same structure but had different physical implementations. Performance improved across problems, demonstrating transfer of knowledge. Neurons in medial prefrontal cortex (mPFC) maintained similar representations across multiple problems, despite their different sensorimotor correlates, whereas hippocampal (dCA1) representations were more strongly influenced by the specifics of each problem. Critically, this was true both for representations of the events that comprised each trial, and those that integrated choices and outcomes over multiple trials to guide subjects decisions. These data suggest that PFC and hippocampus play complementary roles in generalisation of knowledge, with the former abstracting the common structure among related problems, and the latter mapping this structure onto the specifics of the current situation.

neuroscience↗

bioRxiv2021

pyControl: Open source, Python based, hardware and software for controlling behavioural neuroscience experiments.

Laboratory behavioural tasks are an essential research tool. As questions asked of behaviour and brain activity become more sophisticated, the ability to specify and run richly structured tasks becomes more important. An increasing focus on reproducibility also necessitates accurate communication of task logic to other researchers. To these ends we developed pyControl, a system of open source hardware and software for controlling behavioural experiments comprising; a simple yet flexible Python-based syntax for specifying tasks as extended state machines, hardware modules for building behavioural setups, and a graphical user interface designed for efficiently running high throughput experiments on many setups in parallel, all with extensive online documentation. These tools make it quicker, easier and cheaper to implement rich behavioural tasks at scale. As important, pyControl facilitates communication and reproducibility of behavioural experiments through a highly readable task definition syntax and self-documenting features. ResourcesDocumentation: https://pycontrol.readthedocs.io Repositories: https://github.com/pyControl User support: https://groups.google.com/g/pycontrol

neuroscience↗

bioRxiv2019

Distinct roles for DAT and COMT in regulating dopamine transients and reward-guided decision making

Dopamine plays a crucial role in adaptive behavior, and dysfunctional dopamine is implicated in multiple psychiatric conditions characterized by inflexible or inconsistent choices. However, the precise relationship between dopamine and flexible decision making remains unclear. One reason is that, while many studies have focused on the activity of dopamine neurons, efficient dopamine signaling also relies on clearance mechanisms, notably the dopamine transporter (DAT), which predominates in striatum, and catechol-O-methyltransferase (COMT), which predominates in cortex. The exact locus, extent, and timescale of the effects of DAT and COMT are uncertain. Moreover, there is limited data on how acute disruption of either mechanism affects flexible decision making strategies mediated by cortico-striatal networks. To address these issues, we combined pharmacological modulation of DAT and COMT with electrochemistry and behavior in mice. DAT blockade, but not COMT inhibition, regulated sub-second dopamine release in the nucleus accumbens core, but surprisingly neither clearance mechanism affected evoked release in prelimbic cortex. This was not due to a lack of sensitivity, as both amphetamine and atomoxetine changed the kinetics of sub-second release. In a multi-step decision making task where mice had to respond to reversals in either reward probabilities or the choice sequence to reach the goal, DAT blockade selectively impaired, and COMT inhibition improved, performance after reward reversals, but neither manipulation affected the adaptation of choices after actionstate transition reversals. Together, our data suggest that DAT and COMT shape specific aspects of behavioral flexibility by regulating striatal and cortical dopamine, respectively, at fast and slow timescales.

neuroscience↗

bioRxiv2018

pyPhotometry: Open source Python based hardware and software for fiber photometry data acquisition.

Fiber photometry is the process of recording bulk neural activity by measuring fluorescence changes in activity sensitive indicators (e.g. GCaMP) through an optical fiber. We present a system of open source hardware and software for fiber photometry data acquisition consisting of a compact, low cost, data acquisition board built around the Micropython microcontroller, and a cross platform graphical user interface (GUI) for controlling acquisition and visualising signals. The system can acquire two analog and two digital signals, and control two external LEDs via built in LED drivers. Time-division multiplexed illumination allows independent readout of fluorescence evoked by different excitation wavelengths from a single photoreceiver signal. Validation experiments indicate this approach offers better signal to noise for a given average excitation light intensity than sinusoidally-modulated illumination. pyPhotometry is substantially cheaper than commercial hardware filling the same role, and we anticipate, as an open source and comparatively simple tool, it will be easily adaptable and therefore of broad interest to a wide range of users.

neuroscience↗

bioRxiv2017

Single-Trial Inhibition of Anterior Cingulate Disrupts Model-based Reinforcement Learning in a Two-step Decision Task.

The anterior cingulate cortex (ACC) is implicated in learning the value of actions, but it remains poorly understood whether and how it contributes to model-based mechanisms that use action-state predictions and afford behavioural flexibility. To isolate these mechanisms, we developed a multi-step decision task for mice in which both action-state transition probabilities and reward probabilities changed over time. Calcium imaging revealed ramps of choice-selective neuronal activity, followed by an evolving representation of the state reached and trial outcome, with different neuronal populations representing reward in different states. ACC neurons represented the current action-state transition structure, whether state transitions were expected or surprising, and the predicted state given chosen action. Optogenetic inhibition of ACC blocked the influence of action-state transitions on subsequent choice, without affecting the influence of rewards. These data support a role for ACC in model-based reinforcement learning, specifically in using action-state transitions to guide subsequent choice. HighlightsO_LIA novel two-step task disambiguates model-based and model-free RL in mice. C_LIO_LIACC represents all trial events, reward representation is contextualised by state. C_LIO_LIACC represents action-state transition structure, predicted states, and surprise. C_LIO_LIInhibiting ACC impedes action-state transitions from influencing subsequent choice. C_LI

neuroscience↗