Aitken, K. | bioRxiv Science

bioRxiv2022

Neural Population Dynamics of Computing with Synaptic Modulations

In addition to long-time scale rewiring, synapses in the brain are subject to significant modulation that occurs at much shorter time scales and allows them to process short-term information. Despite this, models of the brain like recurrent neural networks (RNNs) often have their weights frozen after training, relying on an internal state stored in neuron activity to process temporal information. Although networks with dynamical synapses have been explored previously, often said dynamics are added to networks that also have recurrent connections and thus the short-time scale computational capabilities of synapse modulation alone remain unclear. In this work, we analyze the dynamics of a network that relies solely on synaptic modulations to process short-time scale information, the multi-plasticity network (MPN). We thoroughly examine the neural population dynamics of the MPN trained on integration-based tasks and compare it to known RNN dynamics, findings the two to have fundamentally different behavior and attractor structure. We find said differences in dynamics allow the MPN to outperform its RNN counterparts on several neuroscience-relevant tasks. Of note, the MPN has a significantly simpler attractor structure that allows it to be more flexible in training and sequential-learning settings. Lastly, how the dynamics change for MPNs trained on contextual and continuous integration tasks is also investigated.

neuroscience↗

bioRxiv2021

The Geometry of Representational Drift in Natural and Artificial Neural Networks

Neurons in sensory areas encode/represent stimuli. Surprisingly, recent studies have suggested that, even during persistent performance, these representations are not stable and change over the course of days and weeks. We examine stimulus representations from fluorescence recordings across hundreds of neurons in the visual cortex using in vivo two-photon calcium imaging and we corroborate previous studies finding that such representations change as experimental trials are repeated across days. This phenomenon has been termed "representational drift". In this study we geometrically characterize the properties of representational drift in the primary visual cortex of mice in two open datasets from the Allen Institute and propose a potential mechanism behind such drift. We observe representational drift both for passively presented stimuli, as well as for stimuli which are behaviorally relevant. Across experiments, the drift most often occurs along directions that have the most variance, leading to a significant turnover in the neurons used for a given representation. Interestingly, despite this significant change due to drift, linear classifiers trained to distinguish neuronal representations show little to no degradation in performance across days. The features we observe in the neural data are similar to properties of artificial neural networks where representations are updated by continual learning in the presence of dropout, i.e. a random masking of nodes/weights, but not other types of noise. Therefore, we conclude that a potential reason for the representational drift in biological networks is driven by an underlying dropout-like noise while continuously learning and that such a mechanism may be computational advantageous for the brain in the same way it is for artificial neural networks, e.g. preventing overfitting.

neuroscience↗