Representational learning of brain responses in executive function and higher-order cognition using deep graph convolutions

Abstract

Brain decoding aims to infer human cognition from recordings of neural activity using modern neuroimaging techniques. Studies so far often concentrated on a limited number of cognitive states and aimed to classifying patterns of brain activity within a local area. This procedure demonstrated a great success on classifying motor and sensory processes but showed limited power over higher cognitive functions. In this work, we investigate a high-order graph convolution model, named ChebNet, to model the segregation and integration organizational principles in neural dynamics, and to decode brain activity across a large number of cognitive domains. By leveraging our prior knowledge on brain organization using a graph-based model, ChebNet graph convolution learns a new representation from task-evoked neural activity, which demonstrates a highly predictive signature of cognitive states and task performance. Our results reveal that between-network integration significantly boosts the decoding of high-order cognition such as visual working memory tasks, while the segregation of localized brain activity is sufficient to classify motor and sensory processes. Using twin and family data from the Human Connectome Project (n = 1,070), we provide evidence that individual variability in the graph representations of working-memory tasks are under genetic control and strongly associated with participants in-scanner behaviors. These findings uncover the essential role of functional integration in brain decoding, especially when decoding high-order cognition other than sensory and motor functions. TeaserO_LIModelling functional integration through graph convolution is a necessary step towards decoding high-order human cognition. C_LI Significance statementOver the past two decades, many studies have applied multivariate pattern analysis to decode what task a human participant is performing, based on a scan of her brain. The vast majority of these studies have however concentrated on select regions and a specific domain, because of the computational complexity of handling full brain data in a multivariate model. With the fast progress in the field of deep learning, it is now possible to decode a variety of cognitive domains simultaneously using a full-brain model. By leveraging our prior knowledge on brain organization using a graph-based model, we uncovered different organizational principles in brain decoding for motor execution and high-order cognition by modelling functional integration through graph convolution.