AUDITORY-MOTOR SURPRISAL REVEALS LEARNING ACROSS MULTIPLE TIMESCALES DURING EXPLORATION AND PRODUCTION

Abstract

Auditory-motor learning is critical for mastering the production of complex sounds, such as speaking and playing music. It is anchored upon internal models of interactions between actions and their sensory consequences, which are fine-tuned by minimizing the errors between the predicted and received sound. Here, we investigated the neural dynamics of sensorimotor learning by manipulating sensorimotor surprisal, with high surprisal defined as unexpected changes in the mapping between actions and their auditory outcomes. Participants performed a piano-playing task in which the key-to-pitch mapping switched unpredictably among three configurations, creating a dynamic sensorimotor environment that required ongoing adaptation. At the change boundaries, a signature of violated motor-to-auditory predictions was found in the auditory evoked responses at N100, which could not be attributed to either purely auditory surprisals or motor execution errors. This prediction error is modulated by short-term context, with greater error responses following longer periods of no map change, indicating that the brain continuously tracks short-term map contexts and rapidly adapts to them. In contrast, 30 minutes of extended goal-directed training on a single key-pitch map modulated P50 amplitude only for the trained map, a modulation that can be explained by a slow, training-driven update of the sensorimotor map. Hence, while auditory predictions from motor actions can be implicitly learned within short-term contexts for rapid adaptation, the complementary process of building a reliable sensorimotor map requires targeted training sustained over time. Our approach of studying auditory-motor surprisal in time-varying sequences reveals that auditory-motor learning is fast, context-sensitive, and shaped by both short- and long-term experience. Significance statementUnderstanding how the brain links motor actions with their sensory consequences is key to explaining how complex skills are acquired and subsequently adapted to changing environments. Yet, the neural mechanisms underpinning the evolution of internal sensorimotor associations across different timescales remain to be elucidated. Here, we extend the concept of surprisal, traditionally used in studies of perception, to the sensorimotor domain, where it is elicited by violations of expectations about the sensory outcome of an action. These expectations are generated by an internal model of sensorimotor associations that must nevertheless remain sufficiently flexible to enable rapid adaptation. Results show that surprisal responses are modulated differently by short-term sensory feedback and longer-term training, via two distinct neural mechanisms underlying adaptation and long-term sensorimotor map updates. These findings advance our understanding of the neural dynamics of sensorimotor learning and inform the design of emerging sensorimotor technologies, enabling novel forms of human-machine interaction and creative applications.