Decoding object categories from EEG during free viewing reveals early information evolution compared to passive viewing

Abstract

Object processing is fundamental to visual perception, and understanding its neural substrates informs many cognitive and computational visual processing models. Thus far, most human studies have used passive viewing paradigms, during which self-driven behavior, such as eye movements, is constrained, and brain activity is evoked by abrupt stimuli onsets. This artificial dissociation of perception and action ignores the natural dynamics of visual processing. Thus, conclusions based on such passive viewing paradigms may not apply to active vision. Here, we study the human neural correlates of category representations during active visual processing by time-locking EEG to self-driven fixations during visual search for natural objects. We combine the deconvolution of overlapping responses to consecutive fixations with multivariate pattern analysis (MVPA) to decode object categories from responses to single fixation. We bridge the active and passive viewing literature by comparing the temporal dynamics of multivariate object representations during free visual search (active viewing) and rapid serial visual presentation (passive viewing), leveraging the high temporal resolution of EEG. We found that categorical information, at different levels of abstraction, can be decoded from single fixations during natural visual processing, and cross-condition decoding revealed that object representations are similar between active and passive viewing conditions. However, representational dynamics emerge significantly earlier in active compared to passive conditions, likely due to the availability of predictive information in free viewing. We highlight methodological considerations for combining MVPA with deconvolution methods. Significance StatementUnderstanding the neural correlates of visual perception is crucial for advancing cognitive and computational models of human vision. This study bridges the gap between passive- and active-vision literature while shedding light on the intricate relationship between perception and action in visual processing. Although eye movements are a fundamental behavior through which visual information is naturally sampled, most neuroimaging studies probe the brain by presenting stimuli abruptly at the center of the screen while participants refrain from moving their eyes. We investigated EEG correlates of visual processing during active visual search and demonstrated that object categories of naturally fixated objects can be decoded from the EEG. We provide novel findings regarding the dynamics of active, compared to passive, visual processing, while contributing to the advancement of EEG analysis methodology.