Biology subjects

Pillow, J. W.

Publications and source records attributed to Pillow, J. W..

At least 19 recordsLinked to original sources

bioRxiv2026

PIANO: Probabilistic Inference Autoencoder Networks for multi-Omics enables robust generative modeling of gene expression and scales single-cell integration to 100 million cells

Single-cell RNA technologies enable the routine acquisition of transcriptomic atlases. However, these molecular profiles are influenced by overlapping sources of variation. Since these covariates confound comparisons, data integration is the first step in most analyses. Three challenges remain: correcting strong batch effects, scaling to millions of cells, and modeling how covariates influence gene expression. To address these challenges, we developed PIANO: Probabilistic Inference Autoencoder Networks for multi-Omics, a deep learning framework whose central feature is a generative model of gene expression data. Additionally, PIANO achieves robust integrations and trains 10x faster than previous methods. PIANO accurately integrates single-cell data across species and across single-cell and spatial transcriptomics modalities. As practical applications, PIANO models spatially-resolved gene expression during Alzheimer's disease progression in human brains and integrates over 100 million cancer cells to model drug perturbations. In summary, PIANO's integration and generative modeling capabilities will empower novel insights for countless future studies.

bioinformatics↗

bioRxiv2025

Fast Optimization of Robust Transcriptomics Embeddings using Probabilistic Inference Autoencoder Networks for multi-Omics

Advances in single-cell genomics technologies enable the routine acquisition of atlases with millions of cells. These datasets often include multiple covariates, such as donors, sequencing platforms, developmental timepoints, and species, which provide new opportunities for discovery and new challenges. To mitigate unwanted sources of variation, dataset integration is the starting point for most analyses. However, existing methods struggle with integrating large complex datasets. To address these limitations, we developed PIANO, a variational autoencoder framework that uses a negative binomial generalized linear model for stronger batch correction, and code compilation for ten times faster training than existing tools. We first demonstrate performant integration compared to commonly used methods on single-species datasets. We then show PIANO enables superior analyses of multiple atlases, solving challenging integration tasks across sequencing platforms, development, and species, while simultaneously preserving desired biological signals. Our contributions include a novel, high-performance integration method and recommendations for integration applications.

neuroscience↗

bioRxiv2025

Improved inference of latent neural states from calcium imaging data

Calcium imaging (CI) is a standard method for recording neural population activity, as it enables simultaneous recording of hundreds-to-thousands of individual somatic signals. Accordingly, CI recordings are prime candidates for population-level latent variable analyses, for example, using models such as Gaussian Process Factor Analysis (GPFA), hidden Markov models (HMMs), and latent dynamical systems. However, these models have been primarily developed and fine-tuned for electrophysiological measurements of spiking activity. To adapt these models for use with the calcium signals recorded with CI, per-neuron fluorescence time-traces are typically either de-convolved to approximate spiking events or analyzed directly under Gaussian observation assumptions. The former approach, while enabling the direct application of latent variable methods developed for spiking data, suffers from the imprecise nature of spike estimation from CI. Moreover, isolated spikes can be undetectable in the fluorescence signal, creating additional uncertainty. A more direct model linking observed fluorescence to latent variables would account for these sources of uncertainty. Here, we develop accurate and tractable models for characterizing the latent structure of neural population activity from CI data. We propose to augment HMM, GPFA, and dynamical systems models with a CI observation model that consists of latent Poisson spiking and autoregressive calcium dynamics. Importantly, this model is both more flexible and directly compatible with standard methods for fitting latent models of neural dynamics. We demonstrate that using this more accurate CI observation model improves latent variable inference and model fitting on both CI observations generated using state-of-the-art biophysical simulations and imaging data recorded in an experimental setting. We expect the developed methods to be widely applicable to many different analyses of population CI data.

neuroscience↗

bioRxiv2025

Inferring learning rules during de novo task learning

Identifying the learning rules that govern behavior is a central problem in neuroscience. While reinforcement learning (RL) offers a unifying theoretical framework, most empirical studies of animal learning behavior have focused on non-stationary environments (e.g. changing reward probabilities in a known task), as opposed to acquiring an entirely new task from scratch. Here we introduce a statistical framework to infer reinforcement learning rules directly from single-animal behavior. Applied to mice learning a perceptual decision-making task, our approach reveals that policy-gradient-like rules capture de novo task learning better than classical temporal-difference algorithms. By fitting flexible parametric learning rules, we uncover systematic deviations from standard RL models, including side-specific learning rates and negative reward baselines. Together, these parameters account for side-biased learning, as well as forgetting and consecutive errors due to aversive responses to incorrect trials. Extending the framework with latent, dynamic learning rates further reveals that animals adapt their learning rates over training and across curricula. These results provide a statistical account of how animals learn from scratch and highlight key departures from classical reinforcement learning algorithms.

neuroscience↗

bioRxiv2025

Learning neural dynamics through instructive signals

Rapid learning is essential for flexible behavior, but its basis in the brain remains unknown. Here we introduce the PRISM plasticity rule, a unifying mechanistic model of three well-established, fast-acting synaptic plasticity rules--in hippocampus, cerebellum and mushroom body--which relies exclusively on pre-synaptic activity and an "instructive signal" from another brain area. Using a multi-region network model we show that guiding PRISM plasticity with instructive signals enables the network to quickly learn extremely flexible nonlinear dynamics underlying behaviorally relevant computations, as well as to emulate unknown external system dynamics from real-time error signals, which we demonstrate with comprehensive simulations supported by exact mathematical theory. Thus, PRISM plasticity guided by instructive signals is well-suited to rapidly learn general-purpose neural computations--in contrast to canonical Hebbian rules. Finally, we show how including this plasticity rule in artificial learning algorithms can solve long-range temporal credit assignment, a long-standing challenge in machine learning. HighlightsO_LIPRISM (PResynaptic and Instructive Signal-Mediated) plasticity--a unifying mechanistic model of three fast-acting plasticity rules in hippocampus, cerebellum, and mushroom body. C_LIO_LIMathematical theory inspired by Support Vector Machine exactly predicts the dynamics learned in a network model through PRISM plasticity. C_LIO_LIExamples of learning nonlinear dynamics in a single shot via sparse instructive signals or from real-time error signals. C_LIO_LIMachine-learning application of instructive signals and PRISM plasticity to solve the long-range temporal credit assignment problem. C_LI

neuroscience↗

bioRxiv2025

Continuous partitioning of neuronal variability

Neurons exhibit substantial trial-to-trial variability in response to repeated stimuli, posing a major challenge for understanding the information content of neural spike trains. In the visual cortex, responses show greater-than-Poisson variability, whose origins and structure remain unclear. To address this puzzle, we introduce a continuous, doubly stochastic model of spike train variability that partitions neural responses into a smooth stimulus-driven component and a time-varying stochastic gain process. We applied this model to spike trains from four visual areas (LGN, V1, V2, and MT) and found that the gain process is well described by an exponentiated power law, with increasing amplitude and slower decay at higher levels of the visual hierarchy. The model also provides analytical expressions for the Fano factor of binned spike counts as a function of timescale, linking observed variability to underlying modulatory dynamics. Together, these results establish a principled framework for characterizing neural variability across cortical processing stages.

neuroscience↗

bioRxiv2024

Aggression experience and observation promote shared behavioral and neural changes

The ability to observe the social behavior of others and use observed information to bias future action is a fundamental building block of social cognition1,2. A foundational question is whether social observation and experience engage common circuit mechanisms that enable behavioral change. While classic studies on social learning have shown that aggressive behaviors can be learned through observation3, it remains unclear whether aggression observation promotes persistent neural changes that generalize to new contexts. Here, to directly compare the effects of aggression experience and observation at brain-wide scale, we develop a strategy to perform large-scale cell-type specific recordings across subcortical networks for social behavior control and learning. We record longitudinally while animals "train" through direct experience or observation, then probe shared differences in behavior and neural activity in a novel "hard" aggression context. Using supervised and unsupervised methods for behavioral quantification, we detect unique signatures of a shared behavioral strategy not present in animals with no training. During observation, we find widespread activation that mimics experience in networks for behavior generation, with critical differences in signals associated with reward and threat learning. After observation, we observe that changes persist into the novel aggression context, with increased similarity in the neural dynamics between experience and observation groups. Network-level modeling reveals persistent shared changes to a core aggression network, with widespread decoupling of inhibition from a key hypothalamic output region. This demonstrates that "experience-like" activity during observation can recruit a shared plasticity mechanism that biases behavior toward adaptive defensive strategies in new contexts.

neuroscience↗

bioRxiv2024

Internal states emerge early during learning of a perceptual decision-making task

Recent work has shown that during perceptual decision-making tasks, animals frequently alternate between different internal states or strategies. However, the question of how or when these emerge during learning remains an important open problem. Does an animal alternate between multiple strategies from the very start of training, or only after extensive exposure to a task? Here we address this question by developing a dynamic latent state model, which we applied to training data from mice learning to perform a visual decision-making task. Remarkably, we found that mice exhibited distinct "engaged" and "biased" states even during early training, with multiple states apparent from the second training session onward. Moreover, our model revealed that the gradual improvement in task performance over the course of training arose from a combination of two factors: (1) increased sensitivity to stimuli across all states; and (2) increased proportion of time spent in a higher-accuracy "engaged" state relative to biased states. These findings highlight the power of our approach for characterizing the temporal evolution of multiple strategies across learning.

neuroscience↗

bioRxiv2024

Modeling Complex Animal Behavior with Latent State Inverse Reinforcement Learning

Understanding complex animal behavior is crucial for linking brain computation to observed actions. While recent research has shifted towards modeling behavior as a dynamic process, few approaches exist for modeling long-term, naturalistic behaviors such as navigation. We introduce discrete Dynamical Inverse Reinforcement Learning (dDIRL), a latent state-dependent paradigm for modeling complex animal behavior over extended periods. dDIRL models animal behavior as being driven by internal state-specific rewards, with Markovian transitions between the distinct internal states. Using expectation-maximization, we infer reward functions corresponding to each internal states and the transition probabilities between them, from observed behavior. We applied dDIRL to water-starved mice navigating a labyrinth, analyzing each animal individually. Our results reveal three distinct internal states sufficient to describe behavior, including a consistent water-seeking state occupied for less than half the time. We also identified two clusters of animals with different exploration patterns in the labyrinth. dDIRL offers a nuanced understanding of how internal states and their associated rewards shape observed behavior in complex environments, paving the way for deeper insights into the neural basis of naturalistic behavior.

neuroscience↗

bioRxiv2024

Active reconfiguration of neural task states

The ability to switch between tasks is a core component of human intelligence, yet a mechanistic understanding of this capacity has remained elusive. Long-standing debates over how task switching is influenced by preparation for upcoming tasks or interference from previous tasks have been difficult to resolve without quantitative neural predictions. We advance this debate by using state-space modeling to directly compare the latent task dynamics in task-optimized recurrent neural networks and human electroencephalographic recordings. Over the inter-trial interval, both networks and brains converged into a neutral task state, a novel control strategy that reconciles the role of preparation and interference in task switching. These findings provide a quantitative account of cognitive flexibility and a promising paradigm for bridging artificial and biological neural networks.

neuroscience↗

bioRxiv2024

Bridging the gap between the connectome and whole-brain activity in C. elegans

A fundamental goal of neuroscience is to understand how anatomy determines the functional properties of the nervous system. However, previous work has failed to show how the functional connections between neurons are derived from the connectome in the nematode C. elegans, raising questions about the extent to which anatomy is informative of signaling1-3. Here, we address this problem using a connectome-constrained dynamical model of the brain, which we fit to whole-brain recordings of neural activity during optogenetic perturbation of single neurons2. This dynamical model, which contains non-zero weights only between anatomically connected neurons, captured causal interactions between all pairs of neurons 82% as well as the reproducibility of the perturbation data themselves. This included interactions between anatomically unconnected neurons, which the model accounted for in terms of signal propagation over paths that include multiple neurons. Strikingly, alternative models fit using a shuffled connectome achieved much lower performance. Finally, we found that adding connections beyond those in the connectome did not improve the models ability to capture causal interactions. Our dynamical model thus provides a link between the connectivity of the C. elegans nervous system and its causal interactions and provides a blueprint for exploring the link between structure and function in other organisms.

neuroscience↗

bioRxiv2024

Disentangling the Roles of Distinct Cell Classes with Cell-Type Dynamical Systems

Latent dynamical systems have been widely used to characterize the dynamics of neural population activity in the brain. However, these models typically ignore the fact that the brain contains multiple cell types. This limits their ability to capture the functional roles of distinct cell classes, and to predict the effects of cell-specific perturbations on neural activity or behavior. To overcome these limitations, we introduce the "cell-type dynamical systems" (CTDS) model. This model extends latent linear dynamical systems to contain distinct latent variables for each cell class, with biologically inspired constraints on both dynamics and emissions. To illustrate our approach, we consider neural recordings with distinct excitatory (E) and inhibitory (I) populations. The CTDS model defines separate latents for both cell types, and constrains the dynamics so that E (I) latents have a strictly positive (negative) effects on other latents. We applied CTDS to recordings from rat frontal orienting fields (FOF) and anterior dorsal striatum (ADS) during an auditory decision-making task. The model achieved higher accuracy than a standard linear dynamical system (LDS), and revealed that the animals choice can be decoded from both E and I latents and thus is not restricted to a single cell-class. We also performed in-silico optogenetic perturbation experiments in the FOF and ADS, and found that CTDS was able to replicate the experimentally observed effects of different perturbations on behavior, whereas a standard LDS model--which does not differentiate between cell types--did not. Crucially, our model allowed us to understand the effects of these perturbations by revealing the dynamics of different cell-specific latents. Finally, CTDS can also be used to identify cell types for neurons whose class labels are unknown in electrophysiological recordings. These results illustrate the power of the CTDS model to provide more accurate and more biologically interpretable descriptions of neural population dynamics and their relationship to behavior.

neuroscience↗

bioRxiv2024

Pre-existing visual responses in a projection-defined dopamine population explain individual learning trajectories

Learning a new task is challenging because the world is high dimensional, with only a subset of features being reward-relevant. What neural mechanisms contribute to initial task acquisition, and why do some individuals learn a new task much more quickly than others? To address these questions, we recorded longitudinally from dopamine (DA) axon terminals in mice learning a visual task. Across striatum, DA responses tracked idiosyncratic and side-specific learning trajectories. However, even before any rewards were delivered, contralateral-side-specific visual responses were present in DA terminals only in the dorsomedial striatum (DMS). These pre-existing responses predicted the extent of learning for contralateral stimuli. Moreover, activation of these terminals improved contralateral performance. Thus, the initial conditions of a projection-specific and feature-specific DA signal help explain individual learning trajectories. More broadly, this work implies that functional heterogeneity across DA projections serves to bias target regions towards learning about different subsets of task features, providing a mechanism to address the dimensionality of the initial task learning problem.

neuroscience↗

bioRxiv2024

Revisiting the high-dimensional geometry of population responses in visual cortex

Recent advances in large-scale recording technology have spurred exciting new inquiries into the high-dimensional geometry of the neural code. However, characterizing this geometry from noisy neural responses, particularly in datasets with more neurons than trials, poses major statistical challenges. We address this problem by developing new tools for the accurate estimation of high-dimensional signal geometry. We apply these tools to investigate the geometry of representations in mouse primary visual cortex. Previous work has argued that these representations exhibit a power law, in which the nth principal component falls off as 1/n. Here we show that response geometry in V1 is better described by a broken power law, in which two different exponents govern the falloff of early and late modes of population activity. Our analysis reveals that later modes decay more rapidly than previously suggested, resulting in a substantially larger fraction of signal variance contained in the early modes of population activity. We examined the signal representations of the early population modes and found them to have higher fidelity than even the most reliable neurons. Intriguingly there are many population modes not captured by classic models of primary visual cortex indicating there is highly redundant yet poorly characterized tuning across neurons. Furthermore, inhibitory neurons tend to co-activate in response to stimuli that drive the early modes consistent with a role in sharpening population level tuning. Overall, our novel and broadly applicable approach overturns prior results and reveals striking structure in a population sensory representation. Significance StatementThe nervous system encodes the visual environment across millions of neurons. Such high-dimensional signals are difficult to estimate--and consequently--to characterize. We address this challenge with a novel statistical method that revises past conceptions of the complexity of encoding in primary visual cortex. We discover population encoding is dominated by approximately ten features while additional features account for much less of the representation than previously thought. Many dominant features are not explained by classic models indicating highly redundant encoding of poorly characterized nonlinear image features. Interestingly, inhibitory neurons respond in unison to dominant features consistent with a role in sharpening population representation. Overall, we discover striking properties of population visual representation with novel, broadly applicable, statistical tools.

neuroscience↗

bioRxiv2024

Identifying the factors governing internal state switches during nonstationary sensory decision-making

Recent work has revealed that mice do not rely on a stable strategy during perceptual decision-making, but switch between multiple strategies within a single session [1, 2]. However, this switching behavior has not yet been characterized in non-stationary environments, and the factors that govern switching remain unknown. Here we address these questions using an internal state model with input-driven transitions. Our approach relies on a hidden Markov model (HMM) with two sets of per-state generalized linear models (GLMs): a set of Bernoulli GLMs for modeling the animals state- and stimulus-dependent choice on each trial, and a multinomial GLM for modeling input-dependent transitions between states. We used this model to analyze a dataset from the International Brain Laboratory (IBL), in which mice performed a binary decision-making task with non-stationary stimulus statistics. We found that mouse behavior in this task was accurately described by a four-state model. This model contained two "engaged" states, in which performance was good despite slight left and right biases, and two "disengaged" states, where performance was low and exhibited with larger left and right biases, respectively. Our analyses revealed that mice preferentially used left-bias strategies during left-bias stimulus blocks, and right-bias strategies during right-bias stimulus blocks, meaning that they could achieve reasonably high performance even in disengaged states simply by biasing choice toward the side with greater prior probability. Our model showed that past choices and past stimuli predicted transitions between left- and right-bias states, while past rewards predicted transitions between engaged and disengaged states. In particular, greater past reward predicted transition to disengaged states, suggesting that disengagement may be associated with satiety.

neuroscience↗

bioRxiv2024

Thoughtful faces: inferring internal states across species using facial features

Animal behaviour is shaped to a large degree by internal cognitive states, but it is unknown whether these states are similar across species. To address this question, here we develop a virtual reality setup in which male mice and macaques engage in the same naturalistic visual foraging task. We exploit the richness of a wide range of facial features extracted from video recordings during the task, to train a Markov-Switching Linear Regression (MSLR). By doing so, we identify, on a singletrial basis, a set of internal states that reliably predicts when the animals are going to react to the presented stimuli. Even though the model is trained purely on reaction times, it can also predict task outcome, supporting the behavioural relevance of the inferred states. The relationship of the identified states to task performance is comparable between mice and monkeys. Furthermore, each state corresponds to a characteristic pattern of facial features that partially overlaps between species, highlighting the importance of facial expressions as manifestations of internal cognitive states across species.

animal behavior and cognition↗

bioRxiv2024

Inferring neural dynamics of memory during naturalistic social communication

Memory processes in complex behaviors like social communication require forming representations of the past that grow with time. The neural mechanisms that support such continually growing memory remain unknown. We address this gap in the context of fly courtship, a natural social behavior involving the production and perception of long, complex song sequences. To study female memory for male song history in unrestrained courtship, we present Natural Continuation (NC)--a general, simulation-based model comparison procedure to evaluate candidate neural codes for complex stimuli using naturalistic behavioral data. Applying NC to fly courtship revealed strong evidence for an adaptive population mechanism for how female auditory neural dynamics could convert long song histories into a rich mnemonic format. Song temporal patterning is continually transformed by heterogeneous nonlinear adaptation dynamics, then integrated into persistent activity, enabling common neural mechanisms to retain continuously unfolding information over long periods and yielding state-of-the-art predictions of female courtship behavior. At a population level this coding model produces multi-dimensional advection-diffusion-like responses that separate songs over a continuum of timescales and can be linearly transformed into flexible output signals, illustrating its potential to create a generic, scalable mnemonic format for extended input signals poised to drive complex behavioral responses. This work thus shows how naturalistic behavior can directly inform neural population coding models, revealing here a novel process for memory formation.

neuroscience↗

bioRxiv2023

Compact deep neural network models of visual cortex

A powerful approach to understanding the computations carried out in visual cortex is to develop models that predict neural responses to arbitrary images. Deep neural network (DNN) models have worked remarkably well at predicting neural responses [1, 2, 3], yet their underlying computations remain buried in millions of parameters. Have we simply replaced one complicated system in vivo with another in silico? Here, we train a data-driven deep ensemble model that predicts macaque V4 responses [~]50% more accurately than currently-used task-driven DNN models. We then compress this deep ensemble to identify compact models that have 5,000x fewer parameters yet equivalent accuracy as the deep ensemble. We verified that the stimulus preferences of the compact models matched those of the real V4 neurons by measuring V4 responses to both maximizing and adversarial images generated using compact models. We then analyzed the inner workings of the compact models and discovered a common circuit motif: Compact models share a similar set of filters in early stages of processing but then specialize by heavily consolidating this shared representation with a precise readout. This suggests that a V4 neurons stimulus preference is determined entirely by its consolidation step. To demonstrate this, we investigated the compression step of a dot-detecting compact model and found a set of simple computations that may be carried out by dot-selective V4 neurons. Overall, our work demonstrates that the DNN models currently used in computational neuroscience are needlessly large; our approach provides a new way forward for obtaining explainable, high-accuracy models of visual cortical neurons.

neuroscience↗