A Minimum Perturbation Theory of Deep Perceptual Learning

Abstract

Perceptual learning (PL) involves long-lasting improvement in perceptual tasks following extensive training and is accompanied by modified neuronal responses in sensory cortical areas in the brain. Understanding the dynamics of PL and the resultant synaptic changes is important for causally connecting PL to the observed neural plasticity. This is theoretically challenging because learning-related changes are distributed across many stages of the sensory hierarchy. In this work, we modeled the sensory hierarchy as a deep nonlinear neural network and studied PL of fine discrimination, a common and well-studied paradigm of PL. Using tools from statistical physics, we developed a mean-field theory of the network in the limit of large number of neurons and large number of examples. Our theory suggests that, in this "thermodynamic" limit, the input-output function of the network can be exactly mapped to that of a deep linear network, allowing us to characterize the space of solutions for the task. Surprisingly, we found that modifications of synaptic weights in lower levels of the hierarchy are both sufficient and necessary for PL. To address the degeneracy of the space of solutions, we postulate that PL dynamics are constrained by a normative "minimum perturbation" (MP) principle, which favors weight matrices with minimal changes relative to their pre-learning values. Interestingly, MP plasticity induces changes to weights and neural representations in all layers of the network, except for the readout weight vector. In addition, such plasticity can be learned simply through slow learning. We further elucidate the properties of MP changes and compare them against experimental findings. Overall, our statistical mechanics theory of PL provides mechanistic and normative understanding of several important empirical findings of PL.