The Listening Effort Profile of Eye Movements: Easy, Difficult, and Impossible Speech Comprehension
Speech comprehension in noisy environments often requires cognitive effort, but listeners may disengage when comprehension becomes impossible. Eye movements have recently emerged as a promising new measure of listening effort, but it remains unclear whether eye movements are sensitive to the full effort profile across easy, difficult, and impossible speech comprehension. Across four experiments, participants listened to sentences at easy, difficult, and impossible levels of multi-talker background babble while pupil size and eye movements were recorded. Pupil size generally followed the expected inverted u-shaped effort profile: low for easy speech, maximal for difficult but still intelligible speech and lower again for impossible speech, although this pattern partly reflected sustained, condition-specific differences and not only sentence-evoked responses. Gaze dispersion - measuring the spread of eye movements - decreased with high temporal selectivity during difficult relative to easy and impossible speech, indicating reduced eye movements during active, effortful listening. However, gaze dispersion was also lower, but less temporally selective, during impossible compared to easy listening, especially in non-baseline-corrected analyses, suggesting that reduced eye movements do not index listening effort uniquely. Instead, eye movements appear to reflect both attentional engagement during difficult listening and disengagement or inward attention when meaningful listening is no longer possible. These findings indicate that pupil size and eye movements provide complementary indices of listening-related cognition, and highlight the integration of listening, cognition, and motor systems.