From sensory to perceptual manifold: the twist of neural geometry

Abstract

Classification constitutes a core cognitive challenge for both biological and artificial intelligence systems, with many tasks potentially reducible to classification problems. Here we investigated how the brain categorizes stimuli that are not linearly separable in the physical world by analyzing the geometry of neural manifolds in high-dimensional neural space, formed by macaques V2 neurons during a classification task on the orientations of motion-induced illusory contours. We identified two related but distinct neural manifolds in this high-dimensional neural space: the sensory and perceptual manifolds. The sensory manifold was embedded in a 3-D subspace defined by three stimulus features, where contour orientations remained linearly inseparable. However, through a series of geometric transformations equivalent to twist operations, this 3-D sensory manifold evolved into a 7-D perceptual manifold with four additional axes, enabling the linear separability of contour orientations. Both formal proof and computational modeling revealed that this dimension expansion was facilitated by nonlinear mixed selectivity neurons exhibiting heterogeneous response profiles. These findings provide insights into the mechanisms by which biological neural networks increase the dimensionality of representational spaces, illustrating how perception arises from sensation through the lens of neural geometry.