FreiPose: A Deep Learning Framework for Precise Animal Motion Capture in 3D Spaces

Abstract

The increasing awareness of the impact of spontaneous movements on neuronal activity has raised the need to track behavior. We present FreiPose, a versatile learning-based framework to directly capture 3D motion of freely definable points with high precision (median error < 3.5% body length, 41.9% improvement compared to state-of-the-art) and high reliability (82.8% keypoints within < 5% body length error boundary, 72.0% improvement). The versatility of FreiPose is demonstrated in two experiments: (1) By tracking freely moving rats with simultaneous electrophysiological recordings in motor cortex, we identified neuronal tuning to behavioral states and individual paw trajectories. (2) We inferred time points of optogenetic stimulation in rat motor cortex from the measured pose across individuals and attributed the stimulation effect automatically to body parts. The versatility and accuracy of FreiPose open up new possibilities for quantifying behavior of freely moving animals and may lead to new ways of setting up experiments.