Learning to represent continuous variables in heterogeneous neural networks

Abstract

Animals must monitor continuous variables such as position or head direction. Manifold attractor networks--which enable a continuum of persistent neuronal states--provide a key framework to explain this monitoring ability. Neural networks with symmetric synaptic connectivity dominate this framework, but are inconsistent with the diverse synaptic connectivity and neuronal representations observed in experiments. Here, we developed a theory for manifold attractors in trained neural networks, which approximate a continuum of persistent states, without assuming unrealistic symmetry. We exploit the theory to predict how asymmetries in the representation and heterogeneity in the connectivity affect the formation of the manifold via training, shape network response to stimulus, and govern mechanisms that possibly lead to destabilization of the manifold. Our work suggests that the functional properties of manifold attractors in the brain can be inferred from the overlooked asymmetries in connectivity and in the low-dimensional representation of the encoded variable.