Evolutionary learning in the brain by heterosynaptic plasticity

Abstract

How the brain modifies synapses to improve the performance of complicated networks remains one of the biggest mysteries in neuroscience. Canonical models suppose synaptic weights change according to pre- and post-synaptic activities (i.e., local plasticity rules), implementing gradient-descent algorithms. However, the lack of experimental evidence to confirm these models suggests that there may be important ingredients neglected by these models. For example, heterosynaptic plasticity, non-local rules mediated by inter-cellular signaling pathways, and the biological implementation of evolutionary algorithms (EA), another machine-learning paradigm that successfully trains large-scale neural networks, are seldom explored. Here we propose and systematically investigate an EA model of brain learning with non-local rules alone. Specifically, a population of agents are represented by different information routes in the brain, whose task performances are evaluated through gating on individual routes alternatively. The selection and reproduction of agents are realized by dopamine-guided heterosynaptic plasticity. Our EA model provides a framework to re-interpret the biological functions of dopamine, meta-plasticity of dendritic spines, memory replay, and the cooperative plasticity between the synapses within a dendritic neighborhood from a new and coherent aspect. Neural networks trained with the model exhibit analogous dynamics to the brain in cognitive tasks. Our EA model manifests broad competence to train spiking or analog neural networks with recurrent or feedforward architecture. Our EA model also demonstrates its powerful capability to train deep networks with biologically plausible binary weights in MNIST classification and Atari-game playing tasks with performance comparable with continuous-weight networks trained by gradient-based methods. Overall, our work leads to a fresh understanding of the brain learning mechanism unexplored by local rules and gradient-based algorithms.