Differential involvement of EEG oscillatory components in sameness vs. spatial-relation visual reasoning tasks

Abstract

The development of deep convolutional networks (DCNs) has recently led to great successes in computer vision and have become de facto computational models of vision. However, a growing body of work suggests that they exhibit critical limitations beyond image categorization. Here, we study a fundamental limitation of DCNs for judging whether two items are the same or different (SD) compared to a baseline assessment of their spatial relationship (SR). We test the prediction that SD tasks recruit additional cortical mechanisms which underlie critical aspects of visual cognition that are not explained by current computational models. We thus recorded EEG signals from 14 participants engaged in the same tasks as the computational models. Importantly, the two tasks were matched in terms of difficulty by an adaptive psychometric procedure: yet, on top of a modulation of evoked potentials, our results revealed higher activity in the low beta (13-20Hz) band in the SD compared to the SR conditions, which we surmise as reflecting the crucial involvement of recurrent mechanisms sustaining working memory and attention. Author SummaryDespite the impressive progress of deep convolutional networks (DCNs) in object recognition, recent studies demonstrated that state-of-the-art vision algorithms encounter severe limitations when performing certain visual reasoning tasks: for instance, convolutional networks can easily solve problems involving spatial relations, but fail in identifying whether two items are identical or different (same-different task). This conclusion led us to test the hypothesis that different computational mechanisms are needed to successfully perform these tasks also in the visual system. First, we confirmed in our simulations that DCNs can successfully perform spatial relationship tasks but struggle with same-different ones. Then, we tested 14 participants on the same experimental design while recording their EEG signals. Remarkably, our results revealed a significant difference between the tasks in the occipital brain regions both in evoked potentials and in the oscillatory dynamics. Specifically, an increase of activity was found when performing the SD over the SR condition. We interpret these results as reflecting the fundamental involvement of recurrent mechanisms implementing cognitive functions such as working memory and attention.