Biology subjects

Paninski, L.

Publications and source records attributed to Paninski, L..

At least 19 recordsLinked to original sources

bioRxiv2026

Linear NeuroPaint: An Efficient Baseline for Cross-Session Neural Alignment and Inpainting

Large-scale Neuropixels recordings sample many brain areas, but no single experimental session records every area of interest. NeuroPaint, a transformer-based autoencoder, integrates such recordings by aligning area-specific latent dynamics across sessions and "inpainting" activity in unrecorded areas. Could a linear model, which would be computationally efficient and potentially more interpretable, suffice to achieve this? To address this question, we introduce Linear NeuroPaint, a linear variant of NeuroPaint, and evaluate it on two multi-area Neuropixels datasets. Linear NeuroPaint achieves strong predictive performance on recorded areas, however, it is less accurate than nonlinear models at cross-area inpainting, despite extensive hyperparameter optimization. Moreover, its parameters are not readily interpretable: incomplete cross-session alignment leads parameter-defined neuron clusters to exhibit heterogeneous response profiles. Together, these results show that linear transformations are insufficient for cross-session alignment and cross-area prediction. Nevertheless, Linear NeuroPaint provides a computationally efficient baseline for benchmarking multi-session neural models.

neuroscience↗

bioRxiv2026

Lightning Pose 3D: an uncertainty-aware framework for data-efficient multi-view animal pose estimation

Multi-view pose estimation is essential for quantifying animal behavior in scientific research, yet current methods struggle to achieve accurate tracking with limited labeled data and suffer from poor uncertainty estimates. We address these challenges with a flexible framework that can operate with or without camera calibration, combining novel training and post-processing techniques with an uncertainty-aware pseudo-labeling distillation procedure. Our multi-view model processes all camera views jointly using a pretrained vision transformer backbone, and a simulated occlusion technique encourages the model to learn robust cross-view correspondences without requiring camera parameters. When camera parameters are available, 3D data augmentations and a triangulation-based loss further encourage geometric consistency. We extend the Ensemble Kalman Smoother (EKS) post-processor to the nonlinear case, leveraging camera geometry, and introduce a variance inflation technique that detects cross-view inconsistencies and corrects overconfident predictions. We validate our approach on five datasets spanning three species (fly, mouse, bird), including a multi-animal dataset with two visually distinct individuals; the proposed pipeline consistently outperforms existing methods across datasets. We demonstrate how these improvements translate to downstream scientific analyses using data from the International Brain Laboratory, showing improved unsupervised behavioral clustering and neural decoding of paw kinematics with just 200 labeled frames. To facilitate adoption, we developed a browser-based, cloud-compatible user interface that supports the full life cycle of multi-view pose estimation, from labeling and model training to post-processing with EKS and diagnostic visualizations.

neuroscience↗

bioRxiv2025

DENDRO: Recovery and denoising of whole-tree dendritic voltagefrom 2D voltage movies

The ability to image voltage at high spatiotemporal resolution across an entire dendritic tree would represent a major advance in systems and circuit neuroscience. Recent advances in genetically encoded voltage indicators (GEVIs) have brought this possibility closer to reality. However, due to fundamental tradeoffs between imaging speed, resolution, SNR, and volume, this goal has remained out of reach. Here we develop a computational method that fuses 3D anatomical information with 2D voltage video data, yielding full time-varying 3D voltage estimates. Our method, termed DENDRO, comprises two steps. In step one, we use the anatomical data to build a microscope model which maps from voltages along the tree to observed fluorescence at the imaging plane. By exploiting local spatial smoothness of the voltage signal, we parameterize the voltage signal using a set of local basis functions, which reduces the dimensionality of the problem and allows us to approximately invert the microscope model. This step leverages spatial but not temporal smoothness of the underlying signal and yields noisy 3D estimates. In step two, we train a lightweight self-supervised neural network to perform spatiotemporal denoising of the inferred voltages. On simulated data, we find that DENDRO is able to recover voltages at high accuracy across an entire dendritic tree. On real voltage movies from hippocampal slices, DENDRO recovers known dendritic phenomena at single trial resolution and millisecond time-scales, and allows visualization of backpropagating action potentials in 3D.

neuroscience↗

bioRxiv2025

Fast photostimulus optimization for holographic control of neural ensemble activity in vivo

Determining the intricate structure and function of neural circuits requires the ability to precisely manipulate circuit activity. Two-photon holographic optogenetics has emerged as a powerful tool for achieving this via flexible excitation of user-defined neural ensembles. However, the precision of two-photon optogenetics has been constrained by off-target stimulation, an effect where proximal non-target neurons can be unintentionally activated due to imperfect spatial confinement of light onto target neurons. Here, we introduce a real-time computational method to mitigating off-target stimulation that first empirically samples each neurons sensitivity to stimulation at proximal locations, and then optimizes stimulation sites using a fast, interpretable model based on adaptive non-negative basis function regression (NBFR). NBFR is highly scalable, completing model fitting for hundreds of neurons in just a few seconds and then optimizing stimulation sites in several hundred milliseconds per stimulus - fast enough for most closed-loop behavioral experiments. We characterize the performance of our approach in both simulations and in vivo experiments in mouse hippocampus, showing its efficacy under realistic experimental conditions. Our results thus establish NBFR-based photostimulus optimization as an important addition to an emerging computational toolkit for precise yet scalable holographic optogenetics.

neuroscience↗

bioRxiv2024

Rarely categorical, always high-dimensional: how the neural code changes along the cortical hierarchy

A long-standing debate in neuroscience concerns whether individual neurons are organized into functionally distinct populations that encode information differently ("categorical" representations [1-3]) and the implications for neural computation. Here, we systematically analyzed how cortical neurons encode cognitive, sensory, and movement variables across 43 cortical regions during a complex task (14,000+ units from the International Brain Laboratory public Brainwide Map data set [4]) and studied how these properties change across the sensory-cognitive cortical hierarchy [5]. We found that the structure of the neural code was scale-dependent: on a whole-cortex scale, neural selectivity was categorical and organized across regions in a way that reflected their anatomical connectivity. However, within individual regions, categorical representations were rare and limited to primary sensory areas, and neuronal responses were instead very diverse. With theoretical arguments and empirical evidence, we demonstrate that the diversity of neural responses enables high-dimensional representations and, hence, high separability, allowing linear readouts to separate experimental conditions in many arbitrary ways. Indeed, when accounting for information that is actually encoded in each area, all cortical regions exhibit maximal separability. Our results indicate that cortical circuits prioritize diversity over categorical structure, supporting a computational regime geared toward high-dimensional, highly-separable neural representations.

neuroscience↗

bioRxiv2024

In vivo cell-type and brain region classification via multimodal contrastive learning

AO_SCPLOWBSTRACTC_SCPLOWCurrent electrophysiological approaches can track the activity of many neurons, yet it is usually unknown which cell-types or brain areas are being recorded without further molecular or histological analysis. Developing accurate and scalable algorithms for identifying the cell-type and brain region of recorded neurons is thus crucial for improving our understanding of neural computation. In this work, we develop a multimodal contrastive learning approach for neural data that can be fine-tuned for different downstream tasks, including inference of cell-type and brain location. We utilize this approach to jointly embed the activity autocorrelations and extracellular waveforms of individual neurons. We demonstrate that our embedding approach, Neuronal Embeddings via MultimOdal contrastive learning (NEMO), paired with supervised fine-tuning, achieves state-of-the-art cell-type classification for two opto-tagged datasets and brain region classification for the public International Brain Laboratory Brain-wide Map dataset. Our method represents a promising step towards accurate cell-type and brain region classification from electrophysiological recordings. The project page and code are available at https://ibl-nemo.github.io/.

neuroscience↗

bioRxiv2024

Exploiting correlations across trials and behavioral sessions to improve neural decoding

Traditional neural decoders model the relationship between neural activity and behavior within individual trials of a single experimental session, neglecting correlations across trials and sessions. However, animals exhibit similar neural activities when performing the same behavioral task, and their behaviors are influenced by past experiences from previous trials. To exploit these informative correlations in large datasets, we introduce two complementary models: a multi-session reduced-rank regression model that shares similar behaviorally-relevant statistical structure in neural activity across sessions to improve decoding, and a multi-session state-space model that shares similar behavioral statistical structure across trials and sessions. Applied across 433 sessions spanning 270 brain regions in the International Brain Laboratory public mouse Neuropixels dataset, our decoders demonstrate improved decoding accuracy for four distinct behaviors compared to traditional approaches. These results generalize across additional datasets, species, and behavioral tasks. Unlike existing deep learning approaches, our models are interpretable and efficient, uncovering low-dimensional representations that predict animal decisions, quantifying single-neuron contributions to decoding behaviors, and identifying different activation timescales of neural activity across the brain. Code: https://github.com/yzhang511/neural_decoding.

neuroscience↗

bioRxiv2024

Coordination and persistence of aggressive visual communication in Siamese fighting fish

Animals coordinate their behavior with each other during both cooperative and agonistic social interactions. Such coordination often adopts the form of "turn taking", in which the interactive partners alternate the performance of a behavior. Apart from acoustic communication, how turn taking between animals is coordinated is not well understood. Furthermore, the neural substrates that regulate persistence in engaging in social interactions are poorly studied. Here, we use Siamese fighting fish (Betta splendens), to study visually-driven turn-taking aggressive behavior. Using encounters with conspecifics and with animations, we characterize the dynamic visual features of an opponent and the behavioral sequences that drive turn taking. Through a brain-wide screen of neuronal activity during coordinated and persistent aggressive behavior, followed by targeted brain lesions, we find that the caudal portion of the dorsomedial telencephalon, an amygdala-like region, promotes persistent participation in aggressive interactions, yet is not necessary for coordination. Our work highlights how dynamic visual cues shape the rhythm of social interactions at multiple timescales, and points to the pallial amygdala as a region controlling engagement in such interactions. These results suggest an evolutionarily conserved role of the vertebrate pallial amygdala in regulating the persistence of emotional states.

neuroscience↗

bioRxiv2024

Distinct anatomical and functional corticospinal inputs innervate different spinal neuron types

The corticospinal tract exerts its influence on movement through spinal neurons, which can be divided into types that exhibit distinct functions. However, it remains unknown whether these functional distinctions are reflected in the corticospinal inputs that different types of spinal neurons receive. Using rabies monosynaptic tracing from individual neuron types in the cervical cord and 3D histological reconstruction in mice, we discovered that different types receive inputs distinctly distributed across cortex, and aligned with cell type function. This included a distinct, sparse distribution of direct inputs from cortex onto motor neurons. Coupling rabies tracing with activity measurement during motor behavior revealed different interneuron types receive different input activity patterns, primarily due to the topographical distribution of the corticospinal neurons contacting them. Our results establish that different spinal neuron types get distinct anatomical and functional inputs from the cortex, and reveal functionally relevant homology to primate corticospinal organization.

neuroscience↗

bioRxiv2024

Identifying Interpretable Latent Factors with Sparse Component Analysis

In many neural populations, the computationally relevant signals are posited to be a set of latent factors - signals shared across many individual neurons. Understanding the relationship between neural activity and behavior requires the identification of factors that reflect distinct computational roles. Methods for identifying such factors typically require supervision, which can be suboptimal if one is unsure how (or whether) factors can be grouped into distinct, meaningful sets. Here, we introduce Sparse Component Analysis (SCA), an unsupervised method that identifies interpretable latent factors. SCA seeks factors that are sparse in time and occupy orthogonal dimensions. With these simple constraints, SCA facilitates surprisingly clear parcellations of neural activity across a range of behaviors. We applied SCA to motor cortex activity from reaching and cycling monkeys, single-trial imaging data from C. elegans, and activity from a multitask artificial network. SCA consistently identified sets of factors that were useful in describing network computations.

neuroscience↗

bioRxiv2023

Towards robust and generalizable representations of extracellular data using contrastive learning

Contrastive learning is quickly becoming an essential tool in neuroscience for extracting robust and meaningful representations of neural activity. Despite numerous applications to neuronal population data, there has been little exploration of how these methods can be adapted to key primary data analysis tasks such as spike sorting or cell-type classification. In this work, we propose a novel contrastive learning framework, CEED (Contrastive Embeddings for Extracellular Data), for high-density extracellular recordings. We demonstrate that through careful design of the network architecture and data augmentations, it is possible to generically extract representations that far outperform current specialized approaches. We validate our method across multiple high-density extracellular recordings. All code used to run CEED can be found at https://github.com/ankitvishnu23/CEED.

neuroscience↗

bioRxiv2023

DREDge: robust motion correction for high-density extracellular recordings across species

High-density microelectrode arrays (MEAs) have opened new possibilities for systems neuroscience in human and non-human animals, but brain tissue motion relative to the array poses a challenge for downstream analyses, particularly in human recordings. We introduce DREDge (Decentralized Registration of Electrophysiology Data), a robust algorithm which is well suited for the registration of noisy, nonstationary extracellular electrophysiology recordings. In addition to estimating motion from spikes in the action potential (AP) frequency band, DREDge enables automated tracking of motion at high temporal resolution in the local field potential (LFP) frequency band. In human intraoperative recordings, which often feature fast (period <1s) motion, DREDge correction in the LFP band enabled reliable recovery of evoked potentials, and significantly reduced single-unit spike shape variability and spike sorting error. Applying DREDge to recordings made during deep probe insertions in nonhuman primates demonstrated the possibility of tracking probe motion of centimeters across several brain regions while simultaneously mapping single unit electrophysiological features. DREDge reliably delivered improved motion correction in acute mouse recordings, especially in those made with an recent ultra-high density probe. We also implemented a procedure for applying DREDge to recordings made across tens of days in chronic implantations in mice, reliably yielding stable motion tracking despite changes in neural activity across experimental sessions. Together, these advances enable automated, scalable registration of electrophysiological data across multiple species, probe types, and drift cases, providing a stable foundation for downstream scientific analyses of these rich datasets.

neuroscience↗

bioRxiv2023

Bypassing spike sorting: Density-based decoding using spike localization from dense multielectrode probes

Neural decoding and its applications to brain computer interfaces (BCI) are essential for understanding the association between neural activity and behavior. A prerequisite for many decoding approaches is spike sorting, the assignment of action potentials (spikes) to individual neurons. Current spike sorting algorithms, however, can be inaccurate and do not properly model uncertainty of spike assignments, therefore discarding information that could potentially improve decoding performance. Recent advances in high-density probes (e.g., Neuropixels) and computational methods now allow for extracting a rich set of spike features from unsorted data; these features can in turn be used to directly decode behavioral correlates. To this end, we propose a spike sorting-free decoding method that directly models the distribution of extracted spike features using a mixture of Gaussians (MoG) encoding the uncertainty of spike assignments, without aiming to solve the spike clustering problem explicitly. We allow the mixing proportion of the MoG to change over time in response to the behavior and develop variational inference methods to fit the resulting model and to perform decoding. We benchmark our method with an extensive suite of recordings from different animals and probe geometries, demonstrating that our proposed decoder can consistently outperform current methods based on thresholding (i.e. multi-unit activity) and spike sorting. Open source code is available at https://github.com/yzhang511/density_decoding.

neuroscience↗

bioRxiv2023

maskNMF: A denoise-sparsen-detect approach for extracting neural signals from dense imaging data

A number of calcium imaging methods have been developed to monitor the activity of large populations of neurons. One particularly promising approach, Bessel imaging, captures neural activity from a volume by projecting within the imaged volume onto a single imaging plane, therefore effectively mixing signals and increasing the number of neurons imaged per pixel. These signals must then be computationally demixed to recover the desired neural activity. Unfortunately, currently-available demixing methods can perform poorly in the regime of high imaging density (i.e., many neurons per pixel). In this work we introduce a new pipeline (maskNMF) for demixing dense calcium imaging data. The main idea is to first denoise and temporally sparsen the observed video; this enhances signal strength and reduces spatial overlap significantly. Next we detect neurons in the sparsened video using a neural network trained on a library of neural shapes. These shapes are derived from segmented electron microscopy images input into a Bessel imaging model; therefore no manual selection of "good" neural shapes from the functional data is required here. After cells are detected, we use a constrained non-negative matrix factorization approach to demix the activity, using the detected cells shapes to initialize the factorization. We test the resulting pipeline on both simulated and real datasets and find that it is able to achieve accurate demixing on denser data than was previously feasible, therefore enabling faithful imaging of larger neural populations. The method also provides good results on more "standard" two-photon imaging data. Finally, because much of the pipeline operates on a significantly compressed version of the raw data and is highly parallelizable, the algorithm is fast, processing large datasets faster than real time.

neuroscience↗

bioRxiv2023

Ultra-high density electrodes improve detection, yield, and cell type specificity of brain recordings

To understand the neural basis of behavior, it is essential to sensitively and accurately measure neural activity at single neuron and single spike resolution. Extracellular electrophysiology delivers this, but it has biases in the neurons it detects and it imperfectly resolves their action potentials. To minimize these limitations, we developed a silicon probe with much smaller and denser recording sites than previous designs, called Neuropixels Ultra (NP Ultra). This device samples neuronal activity at ultra-high spatial density ([~]10 times higher than previous probes) with low noise levels, while trading off recording span. NP Ultra is effectively an implantable voltage-sensing camera that captures a planar image of a neurons electrical field. We use a spike sorting algorithm optimized for these probes to demonstrate that the yield of visually-responsive neurons in recordings from mouse visual cortex improves up to [~]3-fold. We show that NP Ultra can record from small neuronal structures including axons and dendrites. Recordings across multiple brain regions and four species revealed a subset of extracellular action potentials with unexpectedly small spatial spread and axon-like features. We share a large-scale dataset of these brain-wide recordings in mice as a resource for studies of neuronal biophysics. Finally, using ground-truth identification of three major inhibitory cortical cell types, we found that these cell types were discriminable with approximately 75% success, a significant improvement over lower-resolution recordings. NP Ultra improves spike sorting performance, detection of subcellular compartments, and cell type classification to enable more powerful dissection of neural circuit activity during behavior.

neuroscience↗

bioRxiv2023

DARTsort: A modular drift tracking spike sorter for high-density multi-electrode probes

With the advent of high-density, multi-electrode probes, there has been a renewed interest in developing robust and scalable algorithms for spike sorting. Current spike sorting approaches, however, struggle to deal with noisy recordings and probe motion (drift). Here we introduce a modular and interpretable spike sorting pipeline, DARTsort (Drift Aware Registration and Tracking), that builds upon recent advances in denoising, spike localization, and drift estimation. DARTsort integrates a precise estimate of probe drift over time into a model of the spiking signal. This allows our method to be robust to drift across a variety of probe geometries. We show that our spike sorting algorithm outperforms a current state-of-the-art spike sorting algorithm, Kilosort 2.5, on simulated datasets with different drift types and noise levels. Open-source code can be found at https://github.com/cwindolf/dartsort.

neuroscience↗

bioRxiv2023

Removing direct photocurrent artifacts in optogenetic connectivity mapping data via constrained matrix factorization

Monosynaptic connectivity mapping is crucial for building circuit-level models of neural computation. Two-photon optogenetic stimulation, when combined with whole-cell recordings, has the potential to map monosynaptic connectivity at an unprecedented scale. However, optogenetic mapping of nearby connections poses a challenge, due to stimulation artifacts. When the postsynaptic cell expresses opsin, optical excitation can directly induce current in the patched cell, confounding connectivity measurements. This problem is most severe in nearby cell pairs, where synaptic connectivity is often strongest. To overcome this problem, we developed a computational tool, Photocurrent Removal with Constraints (PhoRC). Our method is based on a constrained matrix factorization model which leverages the fact that photocurrent kinetics are consistent across repeated stimulations at similar laser power. We demonstrate on real and simulated data that PhoRC consistently removes photocurrents while preserving synaptic currents, despite variations in photocurrent kinetics across datasets. Our method allows the discovery of synaptic connections which would have been otherwise obscured by photocurrent artifacts, and may thus reveal a more complete picture of synaptic connectivity. PhoRC runs faster than real time and is available at https://github.com/bantin/PhoRC.

neuroscience↗

bioRxiv2023

Brain-wide representations of prior information in mouse decision-making

The neural representations of prior information about the state of the world are poorly understood. To investigate them, we examined brain-wide Neuropixels recordings and widefield calcium imaging collected by the International Brain Laboratory. Mice were trained to indicate the location of a visual grating stimulus, which appeared on the left or right with prior probability alternating between 0.2 and 0.8 in blocks of variable length. We found that mice estimate this prior probability and thereby improve their decision accuracy. Furthermore, we report that this subjective prior is encoded in at least 20% to 30% of brain regions which, remarkably, span all levels of processing, from early sensory areas (LGd, VISp) to motor regions (MOs, MOp, GRN) and high level cortical regions (ACAd, ORBvl). This widespread representation of the prior is consistent with a neural model of Bayesian inference involving loops between areas, as opposed to a model in which the prior is incorporated only in decision-making areas. This study offers the first brain-wide perspective on prior encoding at cellular resolution, underscoring the importance of using large scale recordings on a single standardized task.

neuroscience↗