Shahmohammadi, M. | bioRxiv Science

bioRxiv2020

Informative Neural Codes to Separate Object Categories

In order to develop object recognition algorithms, which can approach human-level recognition performance, researchers have been studying how the human brain performs recognition in the past five decades. This has already in-spired AI-based object recognition algorithms, such as convolutional neural networks, which are among the most successful object recognition platforms today and can approach human performance in specific tasks. However, it is not yet clearly known how recorded brain activations convey information about object category processing. One main obstacle has been the lack of large feature sets, to evaluate the information contents of multiple aspects of neural activations. Here, we compared the information contents of a large set of 25 features, extracted from time series of electroencephalography (EEG) recorded from human participants doing an object recognition task. We could characterize the most informative aspects of brain activations about object categories. Among the evaluated features, event-related potential (ERP) components of N1 and P2a were among the most informative features with the highest information in the Theta frequency bands. Upon limiting the analysis time window, we observed more information for features detecting temporally informative patterns in the signals. The results of this study can constrain previous theories about how the brain codes object category information.

neuroscience↗

bioRxiv2020

Temporal codes provide additional category-related information in object category decoding: a systematic comparison of informative EEG features

How does the human brain encode visual object categories? Our understanding of this has advanced substantially with the development of multivariate decoding analyses. However, conventional electroencephalography (EEG) decoding predominantly use the "mean" neural activation within the analysis window to extract category information. Such temporal averaging overlooks the within-trial neural variability which is suggested to provide an additional channel for the encoding of information about the complexity and uncertainty of the sensory input. The richness of temporal variabilities, however, has not been systematically compared with the conventional "mean" activity. Here we compare the information content of 31 variability-sensitive features against the "mean" of activity, using three independent highly-varied datasets. In whole-trial decoding, the classical event-related potential (ERP) components of "P2a" and "P2b" provided information comparable to those provided by "Original Magnitude Data (OMD)" and "Wavelet Coefficients (WC)", the two most informative variability-sensitive features. In time-resolved decoding, the "OMD" and "WC" outperformed all the other features (including "mean"), which were sensitive to limited and specific aspects of temporal variabilities, such as their phase or frequency. The information was more pronounced in Theta frequency band, previously suggested to support feed-forward visual processing. We concluded that the brain might encode the information in multiple aspects of neural variabilities simultaneously e.g. phase, amplitude and frequency rather than "mean" per se. In our active categorization dataset, we found that more effective decoding of the neural codes corresponds to better prediction of behavioral performance. Therefore, the incorporation of temporal variabilities in time-resolved decoding can provide additional category information and improved prediction of behavior.

neuroscience↗