Multilevel feedback architecture for adaptive regulation of learning in the insect brain
Modulatory (e.g. dopaminergic) neurons provide \"teaching signals\" that drive associative learning across the animal kingdom, but the circuits that regulate their activity and compute teaching signals are still poorly understood. We provide the first synaptic-resolution connectome of the circuitry upstream of all modulatory neurons in a brain center for associative learning, the mushroom body (MB) of the Drosophila larva. We discovered afferent pathways from sensory neurons, as well as an unexpected large population of 61 feedback neuron pairs that provide one- and two-step feedback from MB output neurons. The majority of these feedback pathways link distinct memory systems (e.g. aversive and appetitive). We functionally confirmed some of the structural pathways and found that some modulatory neurons compare inhibitory input from their own compartment and excitatory input from compartments of opposite valence, enabling them to compute integrated common-currency predicted values across aversive and appetitive memory systems. This architecture suggests that the MB functions as an interconnected ensemble during learning and that distinct types of previously formed memories can regulate future learning about a stimulus. We developed a model of the circuit constrained by the connectome and by the functional data which revealed that the newly discovered architectural motifs, namely the multilevel feedback architecture and the extensive cross-compartment connections, increase the computational performance and flexibility on learning tasks. Together our study provides the most detailed view to date of a recurrent brain circuit that computes teaching signals and provides insights into the architectural motifs that support reinforcement learning in a biological system.