Visuoinertial and visual feedback in online steering control

Abstract

Multisensory integration has primarily been studied in static environments, where optimal integration relies on the precision of the respective sensory modalities. However, in numerous situations, sensory information is dynamic and changes over time, due to changes in our bodily state and the surrounding environment. Given that different sensory modalities have different delays, this suggests that optimal integration may not solely depend on sensory precision but may also be affected by the delays associated with each sensory system. To investigate this hypothesis, participants (n = 22, 16 female) engaged in a continuous steering task. Participants sat on a motion platform facing a screen that displayed a cartoonish traffic scene, featuring a car traveling along a road. In the visuoinertial condition, where vestibular and somatosensory feedback were available, they were tasked with counteracting an external multi-frequency perturbation signal, which laterally perturbed the platform and the car, such that the car was kept within the center of the road. In the visual condition, the visual car was perturbed, while the motion platform remained stationary. We show that participants compensate better for the perturbation in the visuoinertial than the visual condition, particularly in the high frequency range of the perturbation. Using computational modelling, we demonstrate that this enhanced performance is partially due to the shorter delay of the vestibular modality. In this condition, participants rely more on the vestibular information, which is less delayed than the more precise but longer delayed, visual information. Author summaryNavigating the world effectively requires the continuous integration of sensory information related to our self-motion, forming a cohesive representation of our current location and direction. Although studies on multisensory integration indicate that sensory inputs are combined optimally based on their precision, these findings have largely been derived from static experimental settings. In real-life situations, the sensory environment changes continuously, as does the state of our body. In such scenarios, how are sensory inputs combined? We developed an experimental paradigm where participants were tasked with controlling a simple vehicle while counteracting a time-varying perturbation signal. To assess the role of different sensory inputs we manipulated whether visual and inertial (primarily vestibular) cues were available or solely visual cues. Our results show that the presence of both visual and inertial cues significantly improved task performance. Modeling suggests this improvement is likely due to the shorter delay associated with the vestibular system compared to the visual system. Collectively, our results suggest that sensory precision alone is insufficient for effective state estimation and control in naturalistic scenarios, where accounting for the different delays of the sensory systems is critical for optimal task performance.