Structural Composition Enables Very Fast Learning

Abstract

AO_SCPLOWBSTRACTC_SCPLOWThere is a gap between neuroscientific theories of learning and the speed of learning observed in many experiments. Since the Cognitive Revolution of the 1950s, compositionality has played a central role in efforts to bridge this gap. Roughly, a compositional system is one where distinct modules are combined according to a set of rules in order to accomplish complex tasks. Recently, significant progress has been made in understanding the emergence of modules in both biological and artificial neural systems. How, and under what conditions, the rules of module recombination are represented in these systems remains an open question. Here we present a neural model that can leverage these rules to dramatically speed up learning. We first show that when faced with multiple tasks which share subcomponents, models learn a low-dimensional representation that captures how subcomponents are reused across the task set. These low-dimensional spaces encode the structure that governs how modules should be recombined. Restricting learning to these subspaces greatly reduces the amount of experience needed to acquire a novel task, even when learning from reinforcement on single trials. In some cases, we can leverage the geometric regularities of these representations to reduce learning to a form of hypothesis testing over a small set of discrete points. Finally, we use this theory to model both behavioral and neural data from non-human primates performing a compositional task, and show that key features in this data are consistent with a model in which exploration during learning is restricted to these low-dimensional spaces. Overall, this work shows that the advantages of modularity in neural systems can be greatly improved upon when models represent the structure of module reuse. Both these features working in tandem lead to learning on timescales similar to biological intelligences, and hence provide a model for how such fast, adaptable behavior can emerge from systems of neurons.