Translating Music to Touch: Exploring Tactile Perception of Pitch, Roughness, and Pleasantness

Abstract

Music is a rich multisensory experience, yet individuals with hearing impairments often lack access to this important aspect of culture. As tactile technologies advance, there is growing interest in whether musical information can be conveyed through vibration. This study investigates how core dimensions of auditory music perception, pitch, roughness, and pleasantness, can be translated into the tactile domain. Participants were asked to rate these perceptual dimensions in response to sinusoidal and complex waveforms, including amplitude-modulated signals, sawtooth, and missing fundamental stimuli. Perceived pitch showed a systematic relationship with stimulus frequency for most participants, suggesting that tactile devices could at least partially convey some form of simple melodic patterns. The sawtooth waveform emerged as particularly effective for representing pitch changes, underscoring the role of rapid temporal transitions in tactile pitch encoding. Roughness ratings were negatively correlated with pleasantness, mirroring well-established findings in auditory perception. Waveforms with sudden temporal changes or rapid amplitude modulations were consistently judged as less pleasant. Taken together, these findings in normal hearing participants may inform the design of vibrotactile displays that could support access to selected music relevant perceptual dimensions, although generalization to people with hearing loss remains to be tested. Importantly, our results highlight the crucial role of the fast temporal envelope rather than the temporal fine structure, in shaping vibrotactile perception. Consistent with this interpretation, our data provide evidence for a tactile analog of the auditory missing fundamental phenomenon, reinforcing the idea that tactile pitch perception primarily relies on envelope periodicity rather than the presence of specific frequency components.