Frequency-Resolved Correlates Of Visual Object Recognition In Human Brain Revealed By Deep Convolutional Neural Networks

Abstract

Previous work demonstrated a direct correspondence between the hierarchy of the human visual areas and layers of deep convolutional neural networks (DCNN) trained on visual object recognition. We used DCNNs to investigate which frequency bands correlate with feature transformations of increasing complexity along the ventral visual pathway. By capitalizing on intracranial depth recordings from 100 patients and 11293 electrodes we assessed the alignment between the DCNN and signals at different frequency bands in different time windows. We found that gamma activity, especially in the low gamma-band (30 - 70 Hz), matched the increasing complexity of visual feature representations in the DCNN. These findings show that the activity of the DCNN captures the essential characteristics of biological object recognition not only in space and time, but also in the frequency domain. These results also demonstrate the potential that modern artificial intelligence algorithms have in advancing our understanding of the brain.\n\nSignificance StatementRecent advances in the field of artificial intelligence have revealed principles about neural processing, in particular about vision. Previous works have demonstrated a direct correspondence between the hierarchy of human visual areas and layers of deep convolutional neural networks (DCNNs), suggesting that DCNN is a good model of visual object recognition in primate brain. Studying intracranial depth recordings allowed us to extend previous works by assessing when and at which frequency bands the activity of the visual system corresponds to the DCNN. Our key finding is that signals in gamma frequencies along the ventral visual pathway are aligned with the layers of DCNN. Gamma frequencies play a major role in transforming visual input to coherent object representations.