EEG mismatch responses in a multi-modal roving stimulus paradigm provide evidence for probabilistic inference across audition, somatosensation and vision.

Abstract

The human brain is constantly subjected to a multi-modal stream of probabilistic sensory inputs. EEG signatures, such as the mismatch negativity (MMN) and the P3, can give valuable insight into neuronal probabilistic inference. Although reported for different modalities, mismatch responses have largely been studied in isolation, with a strong focus on the auditory MMN. To investigate the extent to which early and late mismatch responses across modalities represent comparable signatures of uni- and cross-modal probabilistic inference in the hierarchically structured cortex, we recorded EEG from 32 participants undergoing a novel tri-modal roving stimulus paradigm. The employed sequences consisted of high and low intensity stimuli in the auditory, somatosensory and visual modalities and were governed by uni-modal transition probabilities and cross-modal conditional dependencies. We found modality specific signatures of MMN ([~]100-200ms) in all three modalities, which were source localized to the respective sensory cortices and shared right lateralized pre-frontal sources. Additionally, we identified a cross-modal signature of mismatch processing in the P3a time range ([~]300-350ms), for which a common network with frontal dominance was found. Across modalities, the mismatch responses showed highly comparable parametric effects of stimulus train length, which were driven by standard and deviant response modulations in opposite directions. Strikingly, the P3a responses across modalities were increased for mispredicted compared to predicted and unpredictable stimuli, suggesting sensitivity to cross-modal predictive information. Finally, model comparisons indicated that the observed single trial dynamics were best captured by Bayesian learning models tracking uni-modal stimulus transitions as well as cross-modal conditional dependencies.