Biology subjects

Pillow, J. W.

Publications and source records attributed to Pillow, J. W..

At least 37 records · Page 2Linked to original sources

bioRxiv2023

Flow-field inference from neural data using deep recurrent networks

Computations involved in processes such as decision-making, working memory, and motor control are thought to emerge from the dynamics governing the collective activity of neurons in large populations. But the estimation of these dynamics remains a significant challenge. Here we introduce Flow-field Inference from Neural Data using deep Recurrent networks (FINDR), an unsupervised deep learning method that can infer low-dimensional nonlinear stochastic dynamics underlying neural population activity. Using population spike train data from frontal brain regions of rats performing an auditory decision-making task, we demonstrate that FINDR outperforms existing methods in capturing the heterogeneous responses of individual neurons. We further show that FINDR can discover interpretable low-dimensional dynamics when it is trained to disentangle task-relevant and irrelevant components of the neural population activity. Importantly, the low-dimensional nature of the learned dynamics allows for explicit visualization of flow fields and attractor structures. We suggest FINDR as a powerful method for revealing the low-dimensional task-relevant dynamics of neural populations and their associated computations.

neuroscience↗

bioRxiv2023

From connectome to effectome: learning the causal interaction map of the fly brain

A long-standing goal of neuroscience is to obtain a causal model of the nervous system. This would allow neuroscientists to explain animal behavior in terms of the dynamic interactions between neurons. The recently reported whole-brain fly connectome [1-7] specifies the synaptic paths by which neurons can affect each other but not whether, or how, they do affect each other in vivo. To overcome this limitation, we introduce a novel combined experimental and statistical strategy for efficiently learning a causal model of the fly brain, which we refer to as the "effectome". Specifically, we propose an estimator for a dynamical systems model of the fly brain that uses stochastic optogenetic perturbation data to accurately estimate causal effects and the connectome as a prior to drastically improve estimation efficiency. We then analyze the connectome to propose circuits that have the greatest total effect on the dynamics of the fly nervous system. We discover that, fortunately, the dominant circuits significantly involve only relatively small populations of neurons--thus imaging, stimulation, and neuronal identification are feasible. Intriguingly, we find that this approach also re-discovers known circuits and generates testable hypotheses about their dynamics. Overall, our analyses of the connectome provide evidence that global dynamics of the fly brain are generated by a large collection of small and often anatomically localized circuits operating, largely, independently of each other. This in turn implies that a causal model of a brain, a principal goal of systems neuroscience, can be feasibly obtained in the fly.

neuroscience↗

bioRxiv2022

A new theoretical framework jointly explains behavioral and neural variability across subjects performing flexible decision-making

The ability to flexibly switch our response to external stimuli according to contextual information is critical for successful interactions with a complex world. Context-dependent computations are necessary across many domains1-3, yet their neural implementations remain poorly understood. Here we developed a novel behavioral task in rats to study context-dependent selection and accumulation of evidence for decision-making4-6. Under assumptions supported by both monkey and rat data, we first show mathematically that a network can solve this problem through a combination of three defined components. These components can be identified and tested directly with experimental data. We further show that existing electrophysiological and modeling data are compatible with the full variety of possible combinations of these components, suggesting that different individuals could use different component combinations. To study variability across individual subjects, we developed automated, high-throughput methods to train rats on our task, and we trained many subjects on it. Consistent with theoretical predictions, neural and behavioral analyses revealed substantial heterogeneity across rats, despite uniformly good task performance. Our theory further predicts a specific link between behavioral and neural signatures, which was robustly supported in the data. In summary, our results provide a new experimentally-supported theoretical framework to analyze individual variability in biological and artificial systems performing flexible decision-making tasks, they open the door to cellular-resolution studies of individual variability in higher cognition, and they provide insights into neural mechanisms of context-dependent computation more generally.

neuroscience↗

bioRxiv2022

Temporal integration is a robust feature of perceptual decisions

Making informed decisions in noisy environments requires integrating sensory information over time. However, recent work has suggested that it may be difficult to determine whether an animals decision-making strategy relies on evidence integration or not. In particular, strategies based on extrema-detection or random snapshots of the evidence stream may be difficult or even impossible to distinguish from classic evidence integration. Moreover, such non-integration strategies might be surprisingly common in experiments that aimed to study decisions based on integration. To determine whether temporal integration is central to perceptual decision making, we developed a new model-based approach for comparing temporal integration against alternative "non-integration" strategies for tasks in which the sensory signal is composed of discrete stimulus samples. We applied these methods to behavioral data from monkeys, rats, and humans performing a variety of sensory decision-making tasks. In all species and tasks, we found converging evidence in favor of temporal integration. First, in all observers across studies, the integration model better accounted for standard behavioral statistics such as psychometric curves and psychophysical kernels. Second, we found that sensory samples with large evidence do not contribute disproportionately to subject choices, as predicted by an extrema-detection strategy. Finally, we provide a direct confirmation of temporal integration by showing that the sum of both early and late evidence contributed to observer decisions. Overall, our results provide experimental evidence suggesting that temporal integration is an ubiquitous feature in mammalian perceptual decision-making. Our study also highlights the benefits of using experimental paradigms where the temporal stream of sensory evidence is controlled explicitly by the experimenter, and known precisely by the analyst, to characterize the temporal properties of the decision process.

neuroscience↗

bioRxiv2022

Modeling communication and switching nonlinear dynamics in multi-region neural activity

Understanding how multiple brain regions interact to produce behavior is a major challenge in systems neuroscience, with many regions causally implicated in common tasks such as sensory processing and decision making. However, a precise description of interactions between regions remains an open problem. Moreover, neural dynamics are nonlinear, non-stationary, and can vary dramatically across sessions, days, and animals. Here, we propose multi-region, switching dynamical systems (MR-SDS), a probabilistic model of multiple latent interacting systems that evolve with switching nonlinear dynamics and communication between regions. MR-SDS includes directed interactions between brain regions, allowing for estimation of state-dependent communication signals, and accounts for sensory inputs effects, history effects, and heterogeneity across days and animals. We show that our model accurately recovers latent trajectories, vector fields underlying switching nonlinear dynamics, and cross-region communication profiles in two simulations. We then apply our method to two large-scale, multi-region neural datasets involving mouse decision making. The first includes hundreds of neurons per region, recorded simultaneously at single-cell-resolution across 3 distant cortical regions. The second is a mesoscale widefield dataset of 8 adjacent cortical regions imaged across both hemispheres. On these multi-region datasets, our model outperforms existing piece-wise linear multi-region models and reveals multiple distinct dynamical states and a rich set of cross-region communication profiles.

neuroscience↗

bioRxiv2022

Scalable variational inference for low-rank spatio-temporal receptive fields

An important problem in systems neuroscience is to characterize how a neuron integrates sensory inputs across space and time. The linear receptive field provides a mathematical characterization of this weighting function, and is commonly used to quantify neural response properties and classify cell types. However, estimating receptive fields is difficult in settings with limited data and correlated or high-dimensional stimuli. To overcome these difficulties, we propose a hierarchical model designed to flexibly parameterize low-rank receptive fields. The model includes Gaussian process priors over spatial and temporal components of the receptive field, encouraging smoothness in space and time. We also propose a new temporal prior called temporal relevance determination (TRD), which imposes a variable degree of smoothness as a function of time lag. We derive a scalable algorithm for variational Bayesian inference for both spatial and temporal receptive field components and hyperparameters. The resulting estimator scales to high-dimensional settings in which full-rank maximum likelihood or a posteriori estimates are intractable. We evaluate our approach on neural data from rat retina and primate cortex, and show that it substantially out-performs a variety of existing estimators. Our modeling approach will have useful extensions to a variety of other high-dimensional inference problems with smooth or low-rank structure.

neuroscience↗

bioRxiv2022

One-to-one mapping between deep network units and real neurons uncovers a visual population code for social behavior

The rich variety of behaviors observed in animals arises through the complex interplay between sensory processing and motor control. To understand these sensorimotor transformations, it is useful to build models that predict not only neural responses to sensory input [1, 2, 3, 4, 5] but also how each neuron causally contributes to behavior [6, 7]. Here we demonstrate a novel modeling approach to identify a one-to-one mapping between internal units in a deep neural network and real neurons by predicting the behavioral changes arising from systematic perturbations of more than a dozen neuron types. A key ingredient we introduce is "knockout training", which involves perturb-ing the network during training to match the perturbations of the real neurons during behavioral experiments. We apply this approach to model the sensorimotor transformation of Drosophila melanogaster males during a com-plex, visually-guided social behavior [8, 9, 10]. The visual projection neurons at the interface between the eye and brain form a set of discrete channels, suggesting each channel encodes a single visual feature [11, 12, 13]. Our model reaches a different conclusion: The visual projection neurons form a highly distributed population code that collectively sculpts social behavior. Overall, our framework consolidates behavioral effects elicited from various neural perturbations into a single, unified model, providing a detailed map from stimulus to neuron to behavior.

neuroscience↗

bioRxiv2021

Neural population dynamics underlying evidence accumulation in multiple rat brain regions

Accumulating evidence to make decisions is a core cognitive function. Previous studies have tended to estimate accumulation using either neural or behavioral data alone. Here we develop a unified framework for modeling stimulus-driven behavior and multi-neuron activity simultaneously. We applied our method to choices and neural recordings from three rat brain regions -- the posterior parietal cortex (PPC), the frontal orienting fields (FOF), and the anterior-dorsal striatum (ADS) -- while subjects performed a pulse-based accumulation task. Each region was best described by a distinct accumulation model, which all differed from the model that best described the animals choices. FOF activity was consistent with an accumulator where early evidence was favored while the ADS reflected near perfect accumulation. Neural responses within an accumulation framework unveiled a distinct association between each brain region and choice. Choices were better predicted from all regions using a comprehensive, accumulation-based framework and different brain regions were found to differentially reflect choice-related accumulation signals: FOF and ADS both reflected choice but ADS showed more instances of decision vacillation. Previous studies relating neural data to behaviorally-inferred accumulation dynamics have implicitly assumed that individual brain regions reflect the whole-animal level accumulator. Our results suggest that different brain regions represent accumulated evidence in dramatically different ways and that accumulation at the whole-animal level may be constructed from a variety of neural-level accumulators. Impact StatementA computational framework for combining neural and behavioral data to infer latent dynamics underlying decision-making reveals distinct accumulation dynamics in different brain regions in the rat.

neuroscience↗

bioRxiv2021

Efficient decoding of large-scale neural population responses with Gaussian-process multiclass regression

Neural decoding methods provide a powerful tool for quantifying the information content of neural population codes and the limits imposed by correlations in neural activity. However, standard decoding methods are prone to overfitting and scale poorly to high-dimensional settings. Here, we introduce a novel decoding method to overcome these limitations. Our approach, the Gaussian process multi-class decoder (GPMD), is well-suited to decoding a continuous low-dimensional variable from high-dimensional population activity, and provides a platform for assessing the importance of correlations in neural population codes. The GPMD is a multinomial logistic regression model with a Gaussian process prior over the decoding weights. The prior includes hyperparameters that govern the smoothness of each neurons decoding weights, allowing automatic pruning of uninformative neurons during inference. We provide a variational inference method for fitting the GPMD to data, which scales to hundreds or thousands of neurons and performs well even in datasets with more neurons than trials. We apply the GPMD to recordings from primary visual cortex in three different species: monkey, ferret, and mouse. Our decoder achieves state-of-the-art accuracy on all three datasets, and substantially outperforms independent Bayesian decoding, showing that knowledge of the correlation structure is essential for optimal decoding in all three species.

neuroscience↗

bioRxiv2021

Brain kernel: a new spatial covariance function for fMRI data

A key problem in functional magnetic resonance imaging (fMRI) is to estimate spatial activity patterns from noisy high-dimensional signals. Spatial smoothing provides one approach to regularizing such estimates. However, standard smoothing methods ignore the fact that correlations in neural activity may fall off at different rates in different brain areas, or exhibit discontinuities across anatomical or functional boundaries. Moreover, such methods do not exploit the fact that widely separated brain regions may exhibit strong correlations due to bilateral symmetry or the network organization of brain regions. To capture this non-stationary spatial correlation structure, we introduce the brain kernel, a continuous covariance function for whole-brain activity patterns. We define the brain kernel in terms of a continuous nonlinear mapping from 3D brain coordinates to a latent embedding space, parametrized with a Gaussian process (GP). The brain kernel specifies the prior covariance between voxels as a function of the distance between their locations in embedding space. The GP mapping warps the brain nonlinearly so that highly correlated voxels are close together in latent space, and uncorrelated voxels are far apart. We estimate the brain kernel using resting-state fMRI data, and we develop an exact, scalable inference method based on block coordinate descent to overcome the challenges of high dimensionality (10-100K voxels). Finally, we illustrate the brain kernels usefulness with applications to brain decoding and factor analysis with multiple task-based fMRI datasets.

neuroscience↗

bioRxiv2020

Recurrent dynamics of prefrontal cortex during context-dependent decision-making

A key problem in systems neuroscience is to understand how neural populations integrate relevant sensory inputs during decision-making. Here, we address this problem by training a structured recurrent neural network to reproduce both psychophysical behavior and neural responses recorded from monkey prefrontal cortex during a context-dependent per-ceptual decision-making task. Our approach yields a one-to-one mapping of model neurons to recorded neurons, and explicitly incorporates sensory noise governing the animals performance as a function of stimulus strength. We then analyze the dynamics of the resulting model in order to understand how the network computes context-dependent decisions. We find that network dynamics preserve both relevant and irrelevant stimulus information, and exhibit a grid of fixed points for different stimulus conditions as opposed to a one-dimensional line attractor. Our work provides new insights into context-dependent decision-making and offers a powerful framework for linking cognitive function with neural activity within an artificial model.

neuroscience↗

bioRxiv2020

Mice alternate between discrete strategies during perceptual decision-making

Classical models of perceptual decision-making assume that subjects use a single, consistent strategy to form decisions, or that strategies evolve slowly over time. Here we present new analyses suggesting that this common view is incorrect. We analyzed data from mouse and human decision-making experiments and found that choice behavior relies on an interplay between multiple interleaved strategies. These strategies, characterized by states in a hidden Markov model, persist for tens to hundreds of trials before switching, and may alternate multiple times within a session. The identified mouse decision-making strategies were highly consistent across individuals and comprised a single "engaged" state, in which decisions relied heavily on the sensory stimulus, and several biased states in which errors frequently occurred. These results provide a powerful alternate explanation for "lapses" often observed in rodent psychophysical experiments, and suggest that standard measures of performance mask the presence of dramatic changes in strategy across trials.

neuroscience↗

bioRxiv2020

Identifying signal and noise structure in neuralpopulation activity with Gaussian process factormodels

Neural datasets often contain measurements of neural activity across multiple trials of a repeated stimulus or behavior. An important problem in the analysis of such datasets is to characterize systematic aspects of neural activity that carry information about the repeated stimulus or behavior of interest, which can be considered "signal", and to separate them from the trial-to-trial fluctuations in activity that are not time-locked to the stimulus, which for purposes of such analyses can be considered "noise". Gaussian Process factor models provide a powerful tool for identifying shared structure in high-dimensional neural data. However, they have not yet been adapted to the problem of characterizing signal and noise in multi-trial datasets. Here we address this shortcoming by proposing "signal-noise" Poisson-spiking Gaussian Process Factor Analysis (SNP-GPFA), a flexible latent variable model that resolves signal and noise latent structure in neural population spiking activity. To learn the parameters of our model, we introduce a Fourier-domain black box variational inference method that quickly identifies smooth latent structure. The resulting model reliably uncovers latent signal and trial-to-trial noise-related fluctuations in large-scale recordings. We use this model to show that predominantly, noise fluctuations perturb neural activity within a subspace orthogonal to signal activity, suggesting that trial-by-trial noise does not interfere with signal representations. Finally, we extend the model to capture statistical dependencies across brain regions in multi-region data. We show that in mouse visual cortex, models with shared noise across brain regions out-perform models with independent per-region noise.

neuroscience↗

bioRxiv2020

Extracting the Dynamics of Behavior in Decision-Making Experiments

Understanding how animals update their decision-making behavior over time is an important problem in neuroscience. Decision-making strategies evolve over the course of learning, and continue to vary even in well-trained animals. However, the standard suite of behavioral analysis tools is ill-equipped to capture the dynamics of these strategies. Here, we present a flexible method for characterizing time-varying behavior during decision-making experiments. We show that it successfully captures trial-to-trial changes in an animals sensitivity to not only task-relevant stimuli, but also task-irrelevant covariates such as choice, reward, and stimulus history. We use this method to derive insights from training data collected in mice, rats, and human subjects performing auditory discrimination and visual detection tasks. With this approach, we uncover the detailed evolution of an animals strategy during learning, including adaptation to time-varying task statistics, suppression of sub-optimal strategies, and shared behavioral dynamics between subjects within an experimental population.

neuroscience↗

bioRxiv2019

Poisson balanced spiking networks

An important problem in computational neuroscience is to understand how networks of spiking neurons can carry out various computations underlying behavior. Balanced spiking networks (BSNs) provide a powerful framework for implementing arbitrary linear dynamical systems in networks of integrate-and-fire neurons (Boerlin et al. [1]). However, the classic BSN model requires near-instantaneous transmission of spikes between neurons, which is biologically implausible. Introducing realistic synaptic delays leads to an pathological regime known as "ping-ponging", in which different populations spike maximally in alternating time bins, causing network output to overshoot the target solution. Here we document this phenomenon and provide a novel solution: we show that a network can have realistic synaptic delays while maintaining accuracy and stability if neurons are endowed with conditionally Poisson firing. Formally, we propose two alternate formulations of Poisson balanced spiking networks: (1) a "local" framework, which replaces the hard integrate-and-fire spiking rule within each neuron by a "soft" threshold function, such that firing probability grows as a smooth nonlinear function of membrane potential; and (2) a "population" framework, which reformulates the BSN objective function in terms of expected spike counts over the entire population. We show that both approaches offer improved robustness, allowing for accurate implementation of network dynamics with realistic synaptic delays between neurons. Moreover, both models produce positive correlations between similarly tuned neurons, a feature of real neural populations that is not found in the original BSN. This work unifies balanced spiking networks with Poisson generalized linear models and suggests several promising avenues for future research.

neuroscience↗

bioRxiv2019

Prefrontal cortex exhibits multi-dimensional dynamic encoding during decision-making

Recent work has suggested that prefrontal cortex (PFC) plays a key role in context-dependent perceptual decision-making. Here we investigate population-level coding of decision variables in monkey PFC using a new method for identifying task-relevant dimensions of neural activity. Our analyses reveal that, in contrast to one-dimensional attractor models, PFC has a multi-dimensional code for decisions, context, and relevant as well as irrelevant sensory information. Moreover, these representations evolve in time, with an early linear accumulation phase followed by a phase with rotational dynamics. We identify the dimensions of neural activity associated with these phases, and show that they are not the product of distinct populations, but of a single population with broad tuning characteristics. Finally, we use model-based decoding to show that the transition from linear to rotational dynamics coincides with a sustained plateau in decoding accuracy, revealing that rotational dynamics in PFC preserve sensory as well as choice information for the duration of the stimulus integration period.

neuroscience↗

bioRxiv2019

Neural Anatomy and Optical Microscopy (NAOMi) Simulation for evaluating calcium imaging methods

The past decade has seen a multitude of new in vivo functional imaging methodologies. However, the lack of ground-truth comparisons or evaluation metrics makes large-scale, systematic validation impossible. Here we provide a new framework for evaluating TPM methods via in silico Neural Anatomy and Optical Microscopy (NAOMi) simulation. Our computationally efficient model generates large anatomical volumes of mouse cortex, simulates neural activity, and incorporates optical propagation and scanning to create realistic calcium imaging datasets. We verify NAOMi simulations against in vivo two-photon recordings from mouse cortex. We leverage this access to in silico ground truth to perform direct comparisons between different segmentation algorithms and optical designs. We find modern segmentation algorithms extract strong neural time-courses comparable to estimation using oracle spatial information, but with an increase in the false positive rate. Comparison between optical setups demonstrate improved resilience to motion artifacts in sparsely labeled samples using Bessel beams, increased signal-to-noise ratio and cell-count using low numerical aperture Gaussian beams and nuclear GCaMP, and more uniform spatial sampling with temporal focusing versus multi-plane imaging. Overall, by leveraging the rich accumulated knowledge of neural anatomy and optical physics, we provide a powerful new tool to assess and develop important methods in neural imaging.

neuroscience↗

bioRxiv2018

Detecting and Correcting False Transients in Calcium Imaging

Population recordings of calcium activity are a major source of insight into neural function. Large dataset sizes often require automated methods, but automation can introduce errors that are difficult to detect. Here we show that automatic time course estimation can sometimes lead to significant misattribution errors, in which fluorescence is ascribed to the wrong cell. Misattribution arises when the shapes of overlapping cells are imperfectly defined, or when entire cells or processes are not identified, and misattribution can even be produced by methods specifically designed to handle overlap. To diagnose this problem, we develop a transient-by-transient metric and a visualization tool that allow users to quickly assess the degree of misattribution in large populations. To filter out misattribution, we also design a robust estimator that explicitly accounts for contaminating signals in a generative model. Our methods can be combined with essentially any cell finding technique, empowering users to diagnose and correct at large scale a problem that has the potential to significantly alter scientific conclusions.

neuroscience↗