Topological Augmentation of Latent Information Streams in Feed-Forward Neural Networks

Abstract

Network neuroscience has yielded crucial insights into the systems-level organisation of the brain, however the indirect nature of neuroimaging recordings has rendered the discovery of generative mechanisms for a given function inherently challenging. In parallel, neural network machine-learning models have exhibited breakthrough performance in tackling a range of complex problems, however the principles that govern learning-induced modifications to network structure remain poorly understood, in part due to a lack of analytic tools to quantify the dynamics of network structure. While the question of how network reconfiguration supports learning is mirrored in machine learning and network neuroscience, the different contexts of these fields provides a timely opportunity to bring them together synergistically to investigate the problem. Here we combine these two approaches to reveal connections between the brains network structure and the emerging network structure of an artificial neural network. Specifically, we train a shallow, feedforward neural network to classify handwritten digits and then used a combination of systems neuroscience and information theoretic tools to perform virtual brain analytics on the resultant edge weights and activity patterns of each node. We identify three distinct phases of network reconfiguration across learning, each of which are characterised by unique topological and information-theoretic signatures. Each phase involves aligning the connections of the neural network with patterns of information contained in the input dataset or preceding layers (as relevant). We also observe a process of low-dimensional category separation in the network as a function of learning. Our results offer a systems-level perspective of how artificial neural networks function - in terms of multi-stage reorganization of edge weights and activity patterns to effectively exploit the information content of input data during edge-weight training - while simultaneously enriching our understanding of the methods used by systems neuroscience.