Distinct yet neighboring neural populations encode past, future, and surrounding speech context in the human temporal lobe

Abstract

Context is critical for both human and artificial speech comprehension systems. While the role of preceding context in speech processing has been well documented, the neural mechanisms supporting the integration of subsequent input -- phonemes and words that occur in the future -- remain poorly understood. Here, we leverage advances in artificial speech systems to model the contribution of different sources of context on the neural encoding of speech in the human brain. For neural encoding, context-informed but not context-uninformed speech model embeddings explain unique variance in human neural activity beyond acoustics, including in early speech processing regions. In particular, model embeddings informed by past, future, and surrounding context explain activity in distinct intracranial electrodes. These electrodes are left-lateralised, and spatially intermixed in the temporal lobe. We find that beyond-word context is crucial for the representational quality of speech model embeddings, and in particular for the encoding of abstract linguistic information. Our finding that spatially neighboring yet distinct neural populations in the temporal lobe encode representations shaped by different contextual sources (past, future, and surrounding input) provides key insight into the neural circuitry that integrates multiple forms of contextual information. Furthermore, our results may inform the downstream use of self-supervised speech representations in language technology tasks, and in models of speech comprehension in the human brain.