Biology subjects

Modirshanechi, A.

Publications and source records attributed to Modirshanechi, A..

7 recordsLinked to original sources

bioRxiv2024

Behavioral individuality is a consequence of experience, genetics and learning

Learning and memory are essential components of our individuality. While it is established that behaviour can vary across genetically identical individuals, it remains unknown how much of this variation stems from momentary experience during learning compared to genetics and its past interactions with the environment. To address this, we measured behaviour in thousands of flies from 90 genetic backgrounds while they performed tasks in conditions that either did or did not require learning. Flies that were genetically identical, raised under the same conditions and tested simultaneously in the same environment persistently modified the extent of expressed individuality when they could learn. This learning-induced residual expression of individuality and its dynamics were subdued or absent in innate, learning-independent behaviours. We could quantify and then recreate this phenomenon in computer simulations. The emergence of in silico behavioural individuality was most consistent with the individuality of real flies once we enabled reinforced learning in simulated agents. Moreover, we showed that minor differences in initial conditions of the experiment can exacerbate the expression of individuality within a genotype in a learning-dependent manner. Our results establish that besides the classical G x E interactions shared between individuals in the past, learning from individual momentary experience further extends the expression of individuality.

animal behavior and cognition↗

bioRxiv2024

Representational similarity modulates neural and behavioral signatures of novelty

Novelty signals in the brain modulate learning and drive exploratory behaviors in humans and animals. While the perceived novelty of a stimulus is known to depend on previous experience, the effect of stimulus representations on novelty computation remains elusive. In particular, existing models of novelty computation fail to account for the effects of stimulus similarities that are abundant in naturalistic environments and tasks. Here, we present a unifying, biologically plausible model that captures how stimulus representations modulate novelty signals in the brain and influence novelty-driven learning and exploration. By applying our model to two publicly available data sets, we quantify and explain (i) how generalization across similar visual stimuli affects novelty responses in the mouse visual cortex, and (ii) how generalization across nearby locations impacts mouse exploration in an unfamiliar environment. Our model unifies and explains distinct neural and behavioral signatures of novelty, and enables theory-driven experiment design to investigate the neural mechanisms of novelty computation.

neuroscience↗

bioRxiv2022

Remembering the "When": Hebbian Memory Models for the Time of Past Events

Humans and animals can remember how long ago specific events happened. Little is known about the neural mechanisms that enable remembering the "when" of memories stored for long durations in the episodic memory system - in contrast to interval-timing on the order of seconds and minutes. Based on a systematic exploration of neural coding, association and retrieval schemes, we develop model classes that span the space of possible mechanisms for the reconstruction of the time of past events. In concrete examples we show how network architecture, Hebbian plasticity, synaptic pruning or systems consolidation allow the retrieval of the time of past events. In a simulation, we demonstrate how these mechanisms would enable food caching animals like corvids to remember what they cached, where, and how long ago. To dissociate different hypotheses, we propose three kinds of novel, non-verbal experiments, that can be run with humans and animals. Our simulations predict the experimental results for different classes of models. Our study shows that remembering the "when" can be implemented by many biologically plausible mechanisms and that carefully designed experiments are needed to pin down the actual neural implementation of the memory for the time of past events in different species. Author summaryWas it yesterday, a week ago, or perhaps even longer since I last ate pizza? Questions like these are typically easy to answer, because humans have a good sense of how long ago specific events happened. Research involving animals, such as food-caching birds, has revealed that they, too, can remember the "when" of past events. So far, little is known about the neuronal mechanisms that enable remembering the "when". In this study, we explore systematically concrete hypotheses of how neural circuits, synaptic plasticity and ongoing restructuring of neural circuits allow remembering the "when". We propose experiments, that could be done with humans or animals, to discriminate different biologically plausible mechanisms for remembering the "when".

neuroscience↗

bioRxiv2022

The curse of optimism: a persistent distraction by novelty

Humans successfully explore their environment to find extrinsic rewards, even when exploration requires several intermediate reward-free decisions. It has been hypothesized that intrinsic rewards such as novelty guide this reward-free exploration. However, different intrinsic rewards lead to different exploration strategies, some prone to suboptimal attraction to irrelevant stochastic stimuli, sometimes called the noisy TV problem. Here, we ask whether humans show a similar attraction to reward-free stochasticity and, if so, which type of intrinsic reward guides their exploration. We design a multi-step decision-making paradigm where human participants search for rewarding states in an environment with a highly stochastic but reward-free sub-region. We show that (i) participants persistently explore the stochastic sub-region and (ii) their decisions are best explained by algorithms driven by novelty but not by optimal algorithms driven by information gain. Our results suggest that humans use suboptimal but computationally cheap strategies for exploration in complex environments.

neuroscience↗

bioRxiv2021

Surprise: a unified theory and experimental predictions

Surprising events trigger measurable brain activity and influence human behavior by affecting learning, memory, and decision-making. Currently there is, however, no consensus on the definition of surprise. Here we identify 16 mathematical definitions of surprise in a unifying framework, show how these definitions relate to each other, and prove under what conditions they are indistinguishable. We classify these surprise measures into four main categories: (i) change-point detection surprise, (ii) information gain surprise, (iii) prediction surprise, and (iv) confidence-correction surprise. We design experimental paradigms where different categories make different predictions: we show that surprise-modulation of the speed of learning leads to sensible adaptive behavior only for change-point detection surprise whereas surprise-seeking leads to sensible exploration strategies only for information gain surprise. However, since neither change-point detection surprise nor information gain surprise perfectly reflect the definition of surprise in natural language, a combination of prediction surprise and confidence-correction surprise is needed to capture intuitive aspects of surprise perception. We formalize this combination in a new definition of surprise with testable experimental predictions. We conclude that there cannot be a single surprise measure with all functions and properties previously attributed to surprise. Consequently, we postulate that multiple neural mechanisms exist to detect and signal different aspects of surprise. Author noteAM is grateful to Vasiliki Liakoni, Martin Barry, and Valentin Schmutz for many useful discussions in the course of the last few years, and to Andrew Barto for insightful discussions through and after EPFL Neuro Symposium 2021 on "Surprise, Curiosity and Reward: from Neuroscience to AI". We thank K. Robbins and collaborators for their publicly available experimental data (Robbins et al., 2018). All code needed to reproduce the results reported here will be made publicly available after publication acceptance. This research was supported by Swiss National Science Foundation (no. 200020_184615). Correspondence concerning this article should be addressed to Alireza Modirshanechi, School of Computer and Communication Sciences and School of Life Sciences, EPFL, Lausanne, Switzerland. E-mail: alireza.modirshanechi@epfl.ch.

neuroscience↗

bioRxiv2020

Divergent sensory processing converges in frontal cortex for a planned motor response

The neuronal mechanisms generating a delayed motor response initiated by a sensory cue remain elusive. Here, we tracked the precise sequence of cortical activity in mice transforming a brief whisker stimulus into delayed licking using wide-field calcium imaging, multi-region high-density electrophysiology and time-resolved optogenetic manipulation. Rapid activity evoked by whisker deflection acquired two prominent features for task performance: i) an enhanced excitation of secondary whisker motor cortex, suggesting its important role connecting whisker sensory processing to lick motor planning, and ii) a transient reduction of activity in orofacial sensorimotor cortex, which contributed to suppressing premature licking. Subsequent widespread cortical activity during the delay period largely correlated with anticipatory movements, but when these were accounted for, a focal sustained activity remained in frontal cortex, which was causally essential for licking in the response period. Our results demonstrate key cortical nodes for motor plan generation and timely execution in delayed goal-directed licking.

neuroscience↗

bioRxiv2020

Novelty is not Surprise: Exploration and learning in human sequential decision-making

Classic reinforcement learning (RL) theories cannot explain human behavior in response to changes in the environment or in the absence of external reward. Here, we design a deep sequential decision-making paradigm with sparse reward and abrupt environmental changes. To explain the behavior of human participants in these environments, we show that RL theories need to include surprise and novelty, each with a distinct role. While novelty drives exploration before the first encounter of a reward, surprise increases the rate of learning of a world-model as well as of model-free action-values. Even though the world-model is available for model-based RL, we find that human decisions are dominated by model-free action choices. The world-model is only marginally used for planning but is important to detect surprising events. Our theory predicts human action choices with high probability and allows us to dissociate surprise, novelty, and reward in EEG signals.

neuroscience↗