Biology subjects

Muzy, A.

Publications and source records attributed to Muzy, A..

4 recordsLinked to original sources

bioRxiv2024

Inferring Time-Varying Internal Models of Agents Through Dynamic Structure Learning

Reinforcement learning (RL) models usually assume a stationary internal model structure of agents, which consists of fixed learning rules and environment representations. However, this assumption does not allow accounting for real problem solving by individuals who can exhibit irrational behaviors or hold inaccurate beliefs about their environment. In this work, we present a novel framework called Dynamic Structure Learning (DSL), which allows agents to adapt their learning rules and internal representations dynamically. This structural flexibility enables a deeper understanding of how individuals learn and adapt in real-world scenarios. The DSL framework reconstructs the most likely sequence of agent structures--sourced from a pool of learning rules and environment models--based on observed behaviors. The method provides insights into how an agents internal structure model evolves as it transitions between different structures throughout the learning process. We applied our framework to study rat behavior in a maze task. Our results demonstrate that rats progressively refine their mental map of the maze, evolving from a suboptimal representation associated with repetitive errors to an optimal one that guides efficient navigation. Concurrently, their learning rules transition from heuristic-based to more rational approaches. These findings underscore the importance of both credit assignment and representation learning in complex behaviors. By going beyond simple reward-based associations, our research offers valuable insights into the cognitive mechanisms underlying decision-making in natural intelligence. DSL framework allows better understanding and modeling how individuals in real-world scenarios exhibit a level of adaptability that current AI systems have yet to achieve.

animal behavior and cognition↗

bioRxiv2024

Capturing Optimal and Suboptimal behavior Of Agents Via Structure Learning Of Their Internal Model

This study introduces a novel framework for understanding the cognitive underpinnings of individual behavior which often deviates from rational decision-making aimed at maximizing rewards in real-life scenarios. We propose a structure learning approach to infer an agents internal model, composed of a learning rule and an internal representation of the environment. Crucially, the combined contribution of these components -- rather than their individual contribution -- determines the overall optimality of the agents behavior. By exploring various combinations of learning rules and environment representations, we identify the most probable agent model structure for each individual. We apply this framework to analyze rats learning behavior in a free-choice task within a T-maze with return arms, evaluating different internal models based on optimal and suboptimal learning rules, along with multiple possible representations of the T-maze decision graph. Identifying the most likely agent model structure based on the rats behavioral data reveals that slower learning rats employed either a suboptimal or a moderately optimal agent model, whereas fast learners employ an optimal agent model. Using the inferred agent models for each rat, we explore the qualitative differences in their individual learning processes. Policy entropy, derived from the inferred agent models, also highlights variations in the balance between exploration and exploitation strategies among the rats. Traditional reinforcement learning approaches to addressing suboptimal behavior focus separately on either suboptimal learning rules or flawed environment representations. Our approach jointly models these components, revealing that suboptimal decisions can arise from complex interactions between learning rules and environment representations within an agents internal model. This provides deeper insights into the cognitive mechanisms underlying real-world decision-making.

animal behavior and cognition↗

bioRxiv2023

Coding dynamics of the striatal networks during learning

The rat dorsomedial (DMS) and dorsolateral striatum (DMS), equivalent to caudate nucleus and putamen in primates, are generally required for goal-directed and habit behaviour, respectively. However, it is still unclear whether and how this functional dychotomy emerges in the course of learning. In this study we investigated this issue by recording DMS and DLS single neuron activity in rats performing a continuous spatial alternation task, from the acquisition to optimized performance. We first applied a classical analytical approach to identify task-related activity based on the modifications of single neuron firing rate in relation to specific task events or maze trajectories. We then used an innovative approach based on Hawkes process to reconstruct a directed connectivity graph of simultaneously recorded neurons, that was used to decode animal behavior. This approach enabled us to better unravel the role of DMS and DLS neural networks across learning stages. We showed that DMS and DLS display different task-related activity throughout learning stages, and the proportion of coding neurons over time decreases in the DMS and increases in the DLS. Despite theses major differences, the decoding power of both networks increases during learning. These results suggest that DMS and DLS neural networks gradually reorganize in different ways in order to progressively increase their control over the behavioral performance.

neuroscience↗

bioRxiv2021

Simulation scalability of large brain neuronal networks thanks to time asynchrony

We present a new algorithm to efficiently simulate random models of large neural networks satisfying the property of time asynchrony. The model parameters (average firing rate, number of neurons, synaptic connection probability, and postsynaptic duration) are of the order of magnitude of a small mammalian brain, or of human brain areas. Through the use of activity tracking and procedural connectivity (dynamical regeneration of synapses), both computational and memory complexities of this algorithm are proved to be theoretically linear with the number of neurons. These results are experimentally validated by sequential simulations of millions of neurons and billions of synapses running in few minutes using a single thread of an equivalent desktop computer.

neuroscience↗