Biology subjects

Mihalas, S.

Publications and source records attributed to Mihalas, S..

At least 37 records · Page 2Linked to original sources

bioRxiv2021

The Geometry of Representational Drift in Natural and Artificial Neural Networks

Neurons in sensory areas encode/represent stimuli. Surprisingly, recent studies have suggested that, even during persistent performance, these representations are not stable and change over the course of days and weeks. We examine stimulus representations from fluorescence recordings across hundreds of neurons in the visual cortex using in vivo two-photon calcium imaging and we corroborate previous studies finding that such representations change as experimental trials are repeated across days. This phenomenon has been termed "representational drift". In this study we geometrically characterize the properties of representational drift in the primary visual cortex of mice in two open datasets from the Allen Institute and propose a potential mechanism behind such drift. We observe representational drift both for passively presented stimuli, as well as for stimuli which are behaviorally relevant. Across experiments, the drift most often occurs along directions that have the most variance, leading to a significant turnover in the neurons used for a given representation. Interestingly, despite this significant change due to drift, linear classifiers trained to distinguish neuronal representations show little to no degradation in performance across days. The features we observe in the neural data are similar to properties of artificial neural networks where representations are updated by continual learning in the presence of dropout, i.e. a random masking of nodes/weights, but not other types of noise. Therefore, we conclude that a potential reason for the representational drift in biological networks is driven by an underlying dropout-like noise while continuously learning and that such a mechanism may be computational advantageous for the brain in the same way it is for artificial neural networks, e.g. preventing overfitting.

neuroscience↗

bioRxiv2021

A biologically inspired architecture with switching units can learn to generalize across backgrounds

Humans and other animals navigate different landscapes and environments with ease, a feat that requires the brains ability to rapidly and accurately adapt to different visual domains, generalizing across contexts/backgrounds. Despite recent progress in deep learning applied to classification and detection in the presence of multiple confounds including contextual ones [25, 30], there remain important challenges to address regarding how networks can perform context-dependent computations and how contextually-invariant visual concepts are formed. For instance, recent studies have shown artificial networks that repeatedly misclassified familiar objects set on new backgrounds, e.g. incorrectly labelling known animals when they appeared in a different setting [3]. Here, we show how a bio-inspired network motif can explicitly address this issue. We do this using a novel dataset which can be used as a benchmark for future studies probing invariance to backgrounds. The dataset consists of MNIST digits of varying transparency, set on one of two backgrounds with different statistics: a Gaussian noise or a more naturalistic background from the CIFAR-10 dataset. We use this dataset to learn digit classification when contexts are shown sequentially, and find that both shallow and deep networks have sharply decreased performance when returning to the first background after experience learning the second - the catastrophic forgetting phenomenon in continual learning. To overcome this, we propose an architecture with additional " switching" units that are activated in the presence of a new background. We find that the switching network can learn the new context even with very few switching units, while maintaining the performance in the previous context - but that they must be recurrently connected to network layers. When the task is difficult due to high transparency, the switching network trained on both contexts outperforms networks without switching trained on only one context. The switching mechanism leads to sparser activation patterns, and we provide intuition for why this helps to solve the task. We compare our architecture with other prominent learning methods, and find that elastic weight consolidation is not successful in our setting, while progressive nets are more complex but less effective. Our study therefore shows how a bio-inspired architectural motif can contribute to task generalization across context.

neuroscience↗

bioRxiv2021

A Biologically Plausible Model for Continual Learning using Synaptic Weight Attractors

The human brain readily learns tasks in sequence without forgetting previous ones. Artificial neural networks (ANNs), on the other hand, need to be modified to achieve similar performance. While effective, many algorithms that accomplish this are based on weight importance methods that do not correspond to biological mechanisms. Here we introduce a simple, biologically plausible, method for enabling effective continual learning in ANNs. We show that it is possible to learn a weight-dependent plasticity function that prevents catastrophic forgetting over multiple tasks. We highlight the effectiveness of our method by evaluating it on a set of MNIST classification tasks. We further find that the use of our method promotes synaptic multi-modality, similar to that seen in biology.

neuroscience↗

bioRxiv2021

CNN MouseNet: A biologically constrained convolutional neural network model for mouse visual cortex

Convolutional neural networks trained on object recognition derive inspiration from the neural architecture of the visual system in primates, and have been used as models of the feedforward computation performed in the primate ventral stream. In contrast to the deep hierarchical organization of primates, the visual system of the mouse has a shallower arrangement. Since mice and primates are both capable of visually guided behavior, this raises questions about the role of architecture in neural computation. In this work, we introduce a novel framework for building a biologically constrained convolutional neural network model of the mouse visual cortex. The architecture and structural parameters of the network are derived from experimental measurements, specifically the 100-micrometer resolution interareal connectome, the estimates of numbers of neurons in each area and cortical layer, and the statistics of connections between cortical layers. This network is constructed to support detailed task-optimized models of mouse visual cortex, with neural populations that can be compared to specific corresponding populations in the mouse brain. Using a well-studied image classification task as our working example, we demonstrate the computational capability of this mouse-sized network. Given its relatively small size, MouseNet achieves roughly 2/3rds the performance level on ImageNet as VGG16. In combination with the large scale Allen Brain Observatory Visual Coding dataset, we use representational similarity analysis to quantify the extent to which MouseNet recapitulates the neural representation in mouse visual cortex. Importantly, we provide evidence that optimizing for task performance does not improve similarity to the corresponding biological system beyond a certain point. We demonstrate that the distributions of some physiological quantities are closer to the observed distributions in the mouse brain after task training. We encourage the use of the MouseNet architecture by making the code freely available. Author summaryTask-driven deep neural networks have shown great potential in predicting functional responses of biological neurons. Nevertheless, they are not precise biological analogues, raising questions about how they should be interpreted. Here, we build new deep neural network models of the mouse visual cortex (MouseNet) that are biologically constrained in detail, not only in terms of the basic structure of their connectivity, but also in terms of the count and hence density of neurons within each area, and the spatial extent of their projections. Equipped with the MouseNet model, we can address key questions about mesoscale brain architecture and its role in task learning and performance.We ask, and provide a first set of answers, to: What is the performance of a mouse brain-sized - and mouse brain-structured - model on benchmark image classification tasks? How does the training of a network on this task affect the functional properties of specified layers within the biologically constrained architecture - both overall, and in comparison with recorded function of mouse neurons? We anticipate much future work on allied questions, and the development of more sophisticated models in both mouse and other species, based on the freely available MouseNet model and code which we develop and provide here.

neuroscience↗

bioRxiv2021

A whole-brain monosynaptic input connectome to neuron classes in mouse visual cortex

Identification of the structural connections between neurons is a prerequisite to understanding brain function. We developed a pipeline to systematically map brain-wide monosynaptic inputs to specific neuronal populations using Cre-driver mouse lines and the recombinant rabies tracing system. We first improved the rabies virus tracing strategy to accurately identify starter cells and to efficiently quantify presynaptic inputs. We then mapped brain-wide presynaptic inputs to different excitatory and inhibitory neuron subclasses in the primary visual cortex and seven higher visual areas. Our results reveal quantitative target-, layer- and cell-class-specific differences in the retrograde connectomes, despite similar global input patterns to different neuronal populations in the same anatomical area. The retrograde connectivity we define is consistent with the presence of the ventral and dorsal visual information processing streams and reveals further subnetworks within the dorsal stream. The hierarchical organization of the entire visual cortex can be derived from intracortical feedforward and feedback pathways mediated by upper- and lower-layer input neurons, respectively. This study expands our knowledge of the brain-wide inputs regulating visual areas and demonstrates that our improved rabies virus tracing strategy can be used to scale up the effort in dissecting connectivity of genetically defined cell populations in the whole mouse brain.

neuroscience↗

bioRxiv2021

Backward masking in mice requires visual cortex

Visual masking is used to infer the timescale of conscious perception in humans; yet the underlying circuit mechanisms are not understood. Here we describe a two-alternative choice backward masking task in mice and humans, in which the location of a briefly presented grating is effectively masked within a 50 ms window after stimulus onset. Importantly, human subjects reported reduced perceptual visibility during masking that closely corresponded with behavior deficits. In mice, optogenetic silencing of visual cortex reduces performance over a similar time course as masking. However, response rates and accuracy do not match masking, demonstrating that cortical silencing and masking are distinct phenomena. Spiking responses recorded in primary visual cortex (V1) are consistent with masked behavior when quantified over long, but not short, time windows, indicating masking involves further downstream processing. Behavioral performance can be quantitatively recapitulated by a dual accumulator model constrained by V1 activity. The model and the animals performance for the earliest decisions imply that the initial spike arriving from the periphery can trigger a correct response, but subsequent V1 spikes, evoked by the mask, degrade performance for longer decisions. To test the necessity of visual cortex for backward masking, we optogenetically silenced mask-evoked cortical activity which fully restored discrimination of target location. Together, these results demonstrate that mice, like humans, are susceptible to backward visual masking and that visual cortex has a crucial role in this process.

neuroscience↗

bioRxiv2021

Heterogeneous "cell types" can improve performance of deep neural networks

Deep convolutional neural networks (CNNs) are powerful computational tools for a large variety of tasks (Goodfellow, 2016). Their architecture, composed of layers of repeated identical neural units, draws inspiration from visual neuroscience. However, biological circuits contain a myriad of additional details and complexity not translated to CNNs, including diverse neural cell types (Tasic, 2018). Many possible roles for neural cell types have been proposed, including: learning, stabilizing excitation and inhibition, and diverse normalization (Marblestone, 2016; Gouwens, 2019). Here we investigate whether neural cell types, instantiated as diverse activation functions in CNNs, can assist in the feed-forward computational abilities of neural circuits. Our heterogeneous cell type networks mix multiple activation functions within each activation layer. We assess the value of mixed activation functions by comparing image classification performance to that of homogeneous control networks with only one activation function per network. We observe that mixing activation functions can improve the image classification abilities of CNNs. Importantly, we find larger improvements when the activation functions are more diverse, and in more constrained networks. Our results suggest a feed-forward computational role for diverse cell types in biological circuits. Additionally, our results open new avenues for the development of more powerful CNNs.

neuroscience↗

bioRxiv2021

Geometry of inter-areal interactions in mouse visual cortex

The response of a set of neurons in an area is the result of the sensory input, the interaction of the neurons within the area as well as the long range interactions between areas. We aimed to study the relation between interactions among multiple areas, and if they are fixed or dynamic. The structural connectivity provides a substrate for these interactions, but anatomical connectivity is not known in sufficient detail and it only gives us a static picture. Using the Allen Brain Observatory Visual Coding Neuropixels dataset, which includes simultaneous recordings of spiking activity from up to 6 hierarchically organized mouse cortical visual areas, we estimate the functional connectivity between neurons using a linear model of responses to flashed static grating stimuli. We characterize functional connectivity between populations via interaction subspaces. We find that distinct subspaces of a source area mediate interactions with distinct target areas, supporting the notion that cortical areas use distinct channels to communicate. Most importantly, using a piecewise linear model for activity within each trial, we find that these interactions evolve dynamically over tens of milliseconds following a stimulus presentation. Inter-areal subspaces become more aligned with the intra-areal subspaces during epochs in which a feedforward wave of activity propagates through visual cortical areas. When the short-term dynamics are averaged over, we find that the interaction subspaces are stable over multiple stimulus blocks. These findings have important implications for understanding how information flows through biological neural networks composed of interconnected modules, each of which may have a distinct functional specialization.

neuroscience↗

bioRxiv2021

Learning to Predict in Networks with Heterogeneous and Dynamic Synapses

AO_SCPLOWBSTRACTC_SCPLOWA salient difference between artificial and biological neural networks is the complexity and diversity of individual units in the latter (Tasic et al., 2018). This remarkable diversity is present in the cellular and synaptic dynamics. In this study we focus on the role in learning of one such dynamical mechanism missing from most artificial neural network models, short-term synaptic plasticity (STSP). Biological synapses have dynamics over at least two time scales: a long time scale, which maps well to synaptic changes in artificial neural networks during learning, and the short time scale of STSP, which is typically ignored. Recent studies have shown the utility of such short-term dynamics in a variety of tasks (Masse et al., 2019; Perez-Nieves et al., 2021), and networks trained with such synapses have been shown to better match recorded neuronal activity and animal behavior (Hu et al., 2020). Here, we allow the timescale of STSP in individual neurons to be learned, simultaneously with standard learning of overall synaptic weights. We study learning performance on two predictive tasks, a simple dynamical system and a more complex MNIST pixel sequence. When the number of computational units is similar to the task dimensionality, RNNs with STSP outperform standard RNN and LSTM models. A potential explanation for this improvement is the encoding of activity history in the short-term synaptic dynamics, a biological form of long short-term memory. Beyond a role for synaptic dynamics themselves, we find a reason and a role for their diversity: learned synaptic time constants become heterogeneous across training and contribute to improved prediction performance in feedforward architectures. These results demonstrate how biologically motivated neural dynamics improve performance on the fundamental task of predicting future inputs with limited computational resources, and how learning such predictions drives neural dynamics towards the diversity found in biological brains.

neuroscience↗

bioRxiv2021

Cortical representation variability aligns with in-class variances and can help one-shot learning

Learning invariance across a set of transformations is an important step in mapping high-dimensional inputs to a limited number of classes. After understanding the set of\ invariances, can a new class be learned from one element? We propose a representation which can facilitate such learning: if the variability in representing individual elements across trials aligns with the variability among different elements in a class, then class boundaries learned from the variable representations of one element should be representative of the entire class. In this study, we test whether such a representation occurs in mouse visual systems. We use Neuropixels probes recording single unit activity in mice observing 200 repeats of natural movies taken from a set of 9 continuous clips. We observe that the trial-by-trial variability in the representation of individual frames is well aligned to the variability in representation of multiple frames from the same clip, but not well aligned to the variability among frames from different clips. Thus, the variable representations of images in the mouse cortex can be efficiently used to classify images into their clips. We compare these representations to those in artificial neural networks. We find that, when introducing noise in networks trained for classification (both feed-forward and recurrent networks), the variability in the representation of elements aligns with the in-class variance. The networks which best reproduce the in-vivo observed directions of variability were those trained on a hierarchical classification task. Taken together, these results point to a solution which the cortex can use for one-shot learning of a class: by using noise as a mechanism for generalization. This is a potential computational explanation for the high level of noise observed in the cortex.

neuroscience↗

bioRxiv2020

A solution to temporal credit assignment using cell-type-specific modulatory signals

Animals learn and form memories by jointly adjusting the efficacy of their synapses. How they efficiently solve the underlying temporal credit assignment problem remains elusive. Here, we re-analyze the mathematical basis of gradient descent learning in recurrent spiking neural networks (RSNNs) in light of the recent single-cell transcriptomic evidence for cell-type-specific local neuropeptide signaling in the cortex. Our normative theory posits an important role for the notion of neuronal cell types and local diffusive communication by enabling biologically plausible and efficient weight update. While obeying fundamental biological constraints, including separating excitatory vs inhibitory cell types and observing connection sparsity, we trained RSNNs for temporal credit assignment tasks spanning seconds and observed that the inclusion of local modulatory signaling improved learning efficiency. Our learning rule puts forth a novel form of interaction between modulatory signals and synaptic transmission. Moreover, it suggests a computationally efficient learning method for bio-inspired artificial intelligence.

neuroscience↗

bioRxiv2020

A Convolutional Network Architecture Driven by Mouse Neuroanatomical Data

Convolutional neural networks trained on object recognition derive some inspiration from the neuroscience of the visual system in primates, and have been used as models of the feedforward computation performed in the primate ventral stream. In contrast to the hierarchical organization of primates, the visual system of the mouse has flatter hierarchy. Since mice are capable of visually guided behavior, this raises questions about the role of architecture in neural computation. In this work, we introduce a framework for building a biologically constrained convolutional neural network model of lateral areas of the mouse visual cortex. The structural parameters of the network are derived from experimental measurements, specifically estimates of numbers of neurons in each area and cortical layer, the interareal connec-tome, and the statistics of connections between cortical layers. This network is constructed to support detailed task-optimized models of mouse visual cortex, with neural populations that can be compared to specific corresponding populations in the mouse brain. The code is freely available to support such research.

neuroscience↗

bioRxiv2020

Single circuit in V1 capable of switching contexts during movement using VIP population as a switch

As animals adapt to their environments, their brains are tasked with processing stimuli in different sensory contexts. Whether these computations are context dependent or independent, they are all implemented in the same neural tissue. A crucial question is what neural architectures can respond flexibly to a range of stimulus conditions and switch between them. This is a particular case of flexible architecture that permits multiple related computations within a single circuit. Here, we address this question in the specific case of the visual system circuitry, focusing on context integration, defined as the integration of feedforward and surround information across visual space. We show that a biologically inspired microcircuit with multiple inhibitory cell types can switch between visual processing of the static context and the moving context. In our model, the VIP population acts as the switch and modulates the visual circuit through a disinhibitory motif. Moreover, the VIP population is efficient, requiring only a relatively small number of neurons to switch contexts. This circuit eliminates noise in videos by using appropriate lateral connections for contextual spatio-temporal surround modulation, having superior denoising performance compared to circuits where only one context is learned. Our findings shed light on a minimally complex architecture that is capable of switching between two naturalistic contexts using few switching units. Author SummaryThe brain processes information at all times and much of that information is context-dependent. The visual system presents an important example: processing is ongoing, but the context changes dramatically when an animal is still vs. running. How is context-dependent information processing achieved? We take inspiration from recent neurophysiology studies on the role of distinct cell types in primary visual cortex (V1).We find that relatively few "switching units" -- akin to the VIP neuron type in V1 in that they turn on and off in the running vs. still context and have connections to and from the main population -- is sufficient to drive context dependent image processing. We demonstrate this in a model of feature integration, and in a test of image denoising. The underlying circuit architecture illustrates a concrete computational role for the multiple cell types under increasing study across the brain, and may inspire more flexible neurally inspired computing architectures.

neuroscience↗

bioRxiv2020

Regional, layer, and cell-class specific connectivity of the mouse default mode network

The evolutionarily conserved default mode network (DMN) is characterized by temporally correlated activity between brain regions during resting states. The DMN has emerged as a selectively vulnerable network in multiple disorders, so understanding its anatomical composition will provide fundamental insight into how its function is impacted by disease. Reproducible rodent analogs of the human DMN offer an opportunity to investigate the underlying brain regions and structural connectivity (SC) with high spatial and cell type resolution. Here, we performed systematic analyses using mouse resting state functional magnetic resonance imaging to identify the DMN and whole brain axonal tracing data, co-registered to the 3D Allen Mouse Common Coordinate Framework reference atlas. We identified the specific, predominantly cortical, brain regions comprising the mouse DMN and report preferential SC between these regions. Next, at the cell class level, we report that cortical layer (L) 2/3 neurons in DMN regions project almost exclusively to other DMN regions, whereas L5 neurons project to targets both in and out of the DMN. We then test the hypothesis that in- and out-DMN projection patterns originate from distinct L5 neuron sub-classes using an intersectional viral tracing strategy to label all the axons from neurons defined by a single target. In the ventral retrosplenial cortex, a core DMN region, we found two L5 projection types related to the DMN and mapped them to unique transcriptomically-defined cell types. Together, our results provide a multi-scale description of the anatomical correlates of the mouse DMN.

neuroscience↗

bioRxiv2020

Brain Modeling ToolKit: an Open Source Software Suite for Multiscale Modeling of Brain Circuits

Experimental studies in neuroscience are producing data at a rapidly increasing rate, providing exciting opportunities and formidable challenges to existing theoretical and modeling approaches. To turn massive datasets into predictive quantitative frameworks, the field needs software solutions for systematic integration of data into realistic, multiscale models. Here we describe the Brain Modeling ToolKit (BMTK), a software suite for building models and performing simulations at multiple levels of resolution, from biophysically detailed multi-compartmental, to point-neuron, to population-statistical approaches. Leveraging the SONATA file format and existing software such as NEURON, NEST, and others, BMTK offers consistent user experience across multiple levels of resolution. It permits highly sophisticated simulations to be set up with little coding required, thus lowering entry barriers to new users. We illustrate successful applications of BMTK to large-scale simulations of a cortical area. BMTK is an open-source package provided as a resource supporting modeling-based discovery in the community.

neuroscience↗

bioRxiv2020

Modeling robust and efficient coding in the mouse primary visual cortex using computational perturbations

Investigating how visual inputs are encoded in visual cortex is important for elucidating the roles of cell populations in circuit computations. We here use a recently developed, large-scale model of mouse primary visual cortex (V1) and perturb both single neurons as well as functional- and cell-type defined population of neurons to mimic equivalent optogenetic perturbations. First, perturbations were performed to study the functional roles of layer 2/3 excitatory neurons in inter-laminar interactions. We observed activity changes consistent with the canonical cortical model (Douglas and Martin 1991). Second, single neuron perturbations in layer 2/3 revealed a center-surround inhibition-dominated effect, consistent with recent experiments. Finally, perturbations of multiple excitatory layer 2/3 neurons during visual stimuli of varying contrasts indicated that the V1 model has both efficient and robust coding features. The circuit transitions from predominantly broad like-to-like inhibition at high contrasts to predominantly specific like-to-like excitation at low contrasts. These in silico results demonstrate how the circuit can shift from redundancy reduction to robust codes as a function of stimulus contrast.

neuroscience↗

bioRxiv2020

Adaptation supports short-term memory in a visual change detection task

The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet also produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process. Author SummaryAnimals have to adapt to environments with rich dynamics and maintain multiple types of memories. In this study, we focus on a visual change detection task in mice which requires short-term memory. Learning which features need to be maintained in short-term memory can be realized in a recurrent neural network by changing connections in the network, resulting in memory maintenance through persistent activity. However, in biological networks, a large diversity of time-dependent intrinsic mechanisms are also available. As an alternative to persistent neural activity, we find that learning to make use of internal adapting dynamics better matches both the observed neural activity and behavior of animals in this simple task. The presence of a large diversity of temporal traces could be one of the reasons for the diversity of cells observed. We believe that both learning to keep representations of relevant stimuli in persistent activity and learning to make use of intrinsic time-dependent mechanisms exist, and their relative use will be dependent on the exact task.

neuroscience↗

bioRxiv2020

Autoencoder networks extract latent variables and encode these variables in their connectomes

Spectacular advances in imaging and data processing techniques are revealing a wealth of information about brain connectomes. This raises an exciting scientific opportunity: to infer the underlying circuit function from the structure of its connectivity. A potential roadblock, however, is that - even with well constrained neural dynamics - there are in principle many different connectomes that could support a given computation. Here, we define a tractable setting in which the problem of inferring circuit function from circuit connectivity can be analyzed in detail: the function of input compression and reconstruction, in an autoencoder network with a single hidden layer. Here, in general there is substantial ambiguity in the weights that can produce the same circuit function, because largely arbitrary changes to "input" weights can be undone by applying the inverse modifications to the "output" weights. However, we use mathematical arguments and simulations to show that adding simple, biologically motivated regularization of connectivity resolves this ambiguity in an interesting way: weights are constrained such that the latent variable structure underlying the inputs can be extracted from the weights by using nonlinear dimensionality reduction methods.

neuroscience↗