Multisensory integration of stimulus-driven and goal-driven signals during urgent saccadic choices

Abstract

The ability to orient or attend to sensory events is generally greater in response to visual and auditory cues occurring together than in response to single-modality cues occurring alone. In such cases the perceptual fusion of cross-modal stimuli (multisensory integration) depends on low-level features (e.g., location, intensity) and follows well established principles. However, less is known about multisensory integration mechanisms when behavioral responses are less direct and require top-down control. Here we investigate this in human participants using an urgent multisensory choice task that effectively dissociates stimulus-driven and goal-driven contributions to performance based on their distinct temporal signatures. Task conditions varied the modality of the cues (auditory, visual, or both), their location (left or right), and the rule defining the correct choice (look toward or away from a given cue). When spatially coincident cues were associated with the same response rule ("look away"), we observed multisensory enhancement and performance remained close to a statistical expectation as the choice process unfolded. However, when spatially disparate cues were associated with different rules but the same target, one cue dominated performance and the other produced crossmodal capture, i.e., low-level competition. The results indicate that the efficacy of multisensory integration is dictated by the stimulus-and goal-driven signals produced by each cue, with all four factors rapidly interacting in accordance to the dynamics of spatial attention. Significance StatementAuditory and visual stimuli located near each other in space and time are typically bound into a single sensory percept that draws attention most effectively. However, it is unclear whether such "multisensory integration" occurs during behaviors that go beyond directly attending or orienting to cue stimuli and require top-down control. We investigated this using a novel task design with which stimulus-driven and goal-driven contributions to performance can be accurately identified. We found that multisensory enhancement depends not so much on the complexity of the requested cue-response associations, but rather on the timing and alignment of the stimulus-and goal-driven signals derived from each cue (auditory and visual) -- similar to the way that such signals dictate the allocation of spatial attention.