Shared neural underpinnings of multisensory integration and trial-by-trial perceptual recalibration

Abstract

Multisensory stimuli create behavioral flexibility, e.g. by allowing us to derive a weighted combination of the information received by different senses. They also allow perception to adapt to discrepancies in the sensory world, e.g. by biasing the judgement of unisensory cues based on preceding multisensory evidence. While both facets of multisensory perception are central for behavior, it remains unknown whether they arise from a common neural substrate. In fact, very little is known about the neural mechanisms underlying multisensory perceptual recalibration. To reveal these, we measured whole-brain activity using MEG while human participants performed an audio-visual ventriloquist paradigm designed to reveal multisensory integration within a trial, and the (trial-by-trial) recalibration of subsequent unisensory judgements. Using single trial classification and behavioral modelling, we localized the encoding of sensory information within and between trials, and determined the behavioral relevance of candidate neural representations. While we found neural signatures of perceptual integration within temporal and parietal regions, of these, only medial superior parietal activity retained multisensory information between trials and combined this with current evidence to mediate perceptual recalibration. These results suggest a common neural substrate of sensory integration and trial-by-trial perceptual recalibration, and expose the medial superior parietal cortex as a flexible hub that links present and previous evidence within and between senses to guide behavior.