Movement sonification during haptic exploration shifts emotional outcome without altering texture perception
Adding movement sonification to haptic exploration can change the perceptual outcome of a textured surface through multisensory processing. We hypothesized that auditory-evoked emotions would influence the appraisal of textured surfaces, with corresponding changes reflected in cortical excitability. Participants actively rubbed two different textured surfaces (slippery and rough) either without movement sonification, or with pleasant or disagreeable movement sonification. We found that sounds, whether agreeable or disagreeable, did not change the texture appraisal. However, the less pleasant surface was associated with a stronger negative hedonic valence, particularly when paired with disagreeable movement sonification. Time frequency analyses of EEG activities revealed a significant reduction in beta-band power [15-25 Hz] within the source-estimated sensorimotor and superior posterior parietal cortices when contrasting both pleasant and unpleasant sound with the silent touch. This suggests that the primary somatosensory cortices together with the superior parietal regions participated in the audio-tactile binding, with both pleasant and unpleasant sounds. In addition, we observed a significant increase in beta-band power in medial visual areas, specifically when disagreeable movement sonification was paired with tactile exploration. This may reflect a disengagement of visual cortical processing, potentially amplifying auditory-driven emotional responses and intensifying the perceived unpleasantness of the explored surfaces.