bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2020.08.04.236422

Real-time, low-latency closed-loop feedback using markerless posture tracking

Kane, G. A.·Lopes, G.·Saunders, J. L.·Mathis, A.·Mathis, M. W.
View original source ↗Export metadata ↓

Abstract

The ability to control a behavioral task or stimulate neural activity based on animal behavior in real-time is an important tool for experimental neuroscientists. Ideally, such tools are noninvasive, low-latency, and provide interfaces to trigger external hardware based on posture. Recent advances in pose estimation with deep learning allows researchers to train deep neural networks to accurately quantify a wide variety of animal behaviors. Here we provide a new DeepLabCut-Live! package that achieves low-latency real-time pose estimation (within 15 ms, >100 FPS), with an additional forward-prediction module that achieves zero-latency feedback, and a dynamic-cropping mode that allows for higher inference speeds. We also provide three options for using this tool with ease: (1) a stand-alone GUI (called DLC-Live! GUI), and integration into (2) Bonsai and (3) AutoPilot. Lastly, we benchmarked performance on a wide range of systems so that experimentalists can easily decide what hardware is required for their needs.

Source connections

Explore related subjects

neurosciencePreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Kane, G. A., Lopes, G., Saunders, J. L., Mathis, A., Mathis, M. W.. 2020-08-05. Real-time, low-latency closed-loop feedback using markerless posture tracking. https://doi.org/10.1101/2020.08.04.236422

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Epidermal Langerhans Cells Act as Neuroimmune Regulators of Pain and Itch

Neuroimmune interactions in the skin coordinate immune defense and somatosensory perception, yet how immune cells shape sensory signaling remains poorly defined. Here, we show that Langerhans cells (LCs), epidermal antigen-presenting cells, function as local immune-derived amplifiers of cutaneous sensory signaling. Anatomical and transcriptomic analyses reveal preferential LC association with non-peptidergic sensory neurons. LC activation increases neuronal and behavioral responses to thermal, mechanical, and pruritic stimuli without directly exciting sensory neurons, whereas LC inhibition or ablation selectively reduces pruritogen-evoked itch. We identified semaphorin 7A as a candidate LC-expressed mediator. Semaphorin 7A sensitized cultured sensory neurons to chloroquine and histamine, and neutralizing semaphorin 7A abolished the increase in chloroquine-evoked scratching produced by LC activation. Its receptor PlexinC1 was expressed by most MrgprA3 pruriceptors and blocking PlexinC1 reduced semaphorin 7A-evoked neuronal responses. Together, these findings identify epidermal LCs as local immune-derived amplifiers that regulate the responsiveness of cutaneous sensory circuits and identify semaphorin 7A as a mediator of LC-induced itch potentiation.

neuroscience↗
bioRxiv2026

Feasible closed-loop stimulation for neural latent dynamics control

Modern neuroscience benefits from powerful computational and experimental methods for studying neural populations, including latent dynamics models that predict behavior with few dimensions. But despite their explanatory utility, latent factors are seldom targeted explicitly with experimental perturbations, leaving causal relationships to behavior and inferred dynamical landscapes untested. A promising solution is closed-loop, multi-channel optogenetic control, which, when paired with model predictions, could steer neural trajectories in real time--a feat impossible for traditional, hand-designed stimulation. However, because closed-loop latent control has not been demonstrated in vivo, the hardware and software requirements remain unclear--particularly the necessary stimulation dimensionality and how limited resources such as implant volume and bandwidth should be allocated between stimulation and recording capabilities. Preliminary experiments would thus help guide ongoing technological development. As large-scale exploration of these design spaces in vivo is prohibitively expensive, we conducted a high-throughput simulation survey using biophysically grounded spiking neural networks. We found that low-dimensional actuation is sufficient: in the reference configuration, ~20% of the stimulation channels achieved ~85% of peak control performance, far fewer than the number of neurons. Furthermore, allocating channels roughly equally between stimulation and recording optimized behavioral control, whereas pairing many electrodes with a single light source--the prevailing practice in experimental labs--performed poorly. Even without heavily optimizing modeling or control, we obtained >60% tracking error reduction and >90% evoked behavior accuracy on average for optimal channel counts. These results held across extensive parameter variations, suggesting that closed-loop latent control is feasible with accessible channel counts and establishing design principles for future tool development and in vivo experiments.

neuroscience↗
bioRxiv2026

Identification of Kinome Biomarkers and Therapeutic Rescue in cGMP-Linked Retinal Degeneration

Inherited retinal degenerations, such as Retinitis Pigmentosa (RP) and Leber's congenital amaurosis (LCA), are major causes of progressive and irreversible vision loss. The large number of genes associated with these disorders makes their clinical and molecular characterization difficult, while systemic biomarkers for disease progression or treatment response remain limited. These challenges were addressed by targeting cGMP-associated mechanisms shared by RP and LCA and integrating systemic biomarker profiling with retinal outcomes. Kinase activity analysis (kinome analysis) in peripheral blood mononuclear cells distinguished healthy and diseased states and captured therapy-associated signatures in rd10 mice treated with LP-CN03, a liposomal formulation of the cGMP analog CN03. This first systemic analysis in a therapeutically responsive animal model highlights kinome analysis as a potential translational readout. Given shared cGMP signaling in RP and AIPL1-associated LCA, LP-CN03's therapeutic potential was further evaluated in a new CRISPR-Cas9-generated Aipl1-/- mouse model. Treatment with LP-CN03 preserved retinal structure and function, as shown by electroretinography, longitudinal optical coherence tomography, fundus imaging, optomotor testing, and photoreceptor-specific gene expression. Together, these findings provide independent in vivo evidence for the neuroprotective effect of LP-CN03 in both cGMP-related RP and LCA, also highlighting the translational value of kinome profiling.

neuroscience↗