Biology subjects

Brennan, J. R.

Publications and source records attributed to Brennan, J. R..

4 recordsLinked to original sources

bioRxiv2026

Visual speech enhances phoneme separability in human superior temporal gyrus

Visual speech, such as lipreading, facilitates spoken word recognition, but the neural mechanisms underlying audiovisual speech perception remain poorly understood. Visual cues may disambiguate fine-grained articulatory features during early perceptual stages or instead integrate with speech at more categorical, phoneme-level stages. To test how speech representations are modulated by visual input, we analyzed intracranial electroencephalography (iEEG) signals recorded from 12 epilepsy patients performing an audiovisual speech perception task. Participants perceived 16 monosyllabic words presented in auditory-only, visual-only, or congruent audiovisual formats. Words were constructed from four onset consonants (/b/, /g/, /m/, /n/) and four rimes (vowel nucleus and any coda consonants). We examined event-related potentials (ERP) in superior temporal gyrus (STG) and trained support vector machine (SVM) classifiers to decode word identity from neural activity at individual electrodes. Discrete and continuous confusion matrices captured complementary changes in classification accuracy and normalized inverse classification loss, a continuous proxy for classifier confidence. Decoding performance was hierarchically evaluated at the word, phoneme, and phonetic feature levels to determine the representations affected by visual speech. Congruent audiovisual speech increased classifier confidence for phoneme-level representations and improved decoding accuracy at both the word and phoneme level, without corresponding effects on phonetic features. Time-resolved analyses further revealed earlier successful decoding for audiovisual than auditory-only speech, with audiovisual enhancement primarily observed for onset consonants rather than rimes. Together, these findings suggest that visual speech sharpens primarily categorical phoneme representations in STG, with accelerated speech processing and improved word recognition emerging as downstream consequences of phoneme-level enhancement.

neuroscience↗

bioRxiv2026

Beyond next-word prediction: hierarchical linguistic composition drives LLM-brain alignment in time

The internal representations of large language models (LLMs) correlate, or "align", with human neural activity during language comprehension. One view holds that this alignment reflects shared sensitivity to statistical patterns in LLMs and humans, while others hold that it reflects, at least in part, the emergence of shared linguistic representations in these systems. Here, we investigate whether hierarchical linguistic composition, a property believed to be fundamental to human language, modulates LLM-brain alignment. To this end, we manipulated syntax, compositional semantics, and associative semantics in English sentences that were presented to both an LLM and human participants during an electroencephalography (EEG) experiment. We matched linguistically manipulated stimuli in predictability, which allows us to tease apart alignment induced by linguistic structure from statistical factors. By comparing LLM-EEG alignment scores that were derived using a linear encoding model across predictability-matched conditions, we evaluate how linguistic manipulations modulate the alignment between human EEG reading data and contextual embeddings extracted word-by-word from the hidden layers of GPT2-XL. Three key patterns emerge: (1) increased alignment for word sequences with syntactic structure, (2) decreased alignment for sentences with compositional semantics, and (3) associative semantics does not modulate alignment. These observed linguistic modulations of LLM-EEG alignment take place above and beyond predictability. Our results indicate that associative semantics is encoded similarly by LLMs and the brain, as are at least some aspects of syntactic structure, while compositional semantics is more uniquely encoded in the human brain.

neuroscience↗

bioRxiv2024

Decoding the Neural Dynamics of Headed Syntactic Structure Building

The brain builds hierarchical phrases during language comprehension; however, the representational details and dynamics of the phrase-building process remain underspecified. This study directly probes whether the neural code of a phrase involves reactivating the syntactic property of a key subcomponent (the "phrasal head"). To this end, we train a part-of-speech sliding-window neural decoder (verb vs. adverb) on EEG signals recorded while participants (N = 30) read sentences in a controlled experiment. The decoder reaches above-chance performance that is spatiotemporally consistent and generalizes to unseen data across sentence positions. Appling the decoder to held-out data yields predicted activation levels for the verbal "head" of a verb phrase at a distant non-head word (adverb); the critical adverb appeared either at the end of a verb phrase or at a sequentially and lexically matched position with no verb phrase boundary. There is stronger verb activation beginning at [~]600 milliseconds at the critical adverb when it appears at a verb phrase boundary; this effect is not modulated by the strength of conceptual association between the two subcomponents in the verb phrase nor does it reflect word predictability. Time-locked analyses additionally reveal a negativity waveform component and increased beta-delta inter-trial phase coherence, both previously linked to linguistic composition, in a similar time window. With a novel application of neural decoding, our findings delineate the temporal dynamics by which the brain encodes phrasal representations by, in part, reactivating the representation of key subcomponents. We thus establish a link between cognitive accounts of phrase structure representations and electrophysiological dynamics.

neuroscience↗

bioRxiv2021

Le Petit Prince: A multilingual fMRI corpus using ecological stimuli

Neuroimaging using more ecologically valid stimuli such as audiobooks has advanced our understanding of natural language comprehension in the brain. However, prior naturalistic stimuli have typically been restricted to a single language, which limited generalizability beyond small typological domains. Here we present the Le Petit Prince fMRI Corpus (LPPC-fMRI), a multilingual resource for research in the cognitive neuroscience of speech and language during naturalistic listening (Open-Neuro: ds003643). 49 English speakers, 35 Chinese speakers and 28 French speakers listened to the same audiobook The Little Prince in their native language while multi-echo functional magnetic resonance imaging was acquired. We also provide time-aligned speech annotation and word-by-word predictors obtained using natural language processing tools. The resulting timeseries data are shown to be of high quality with good temporal signal-to-noise ratio and high inter-subject correlation. Data-driven functional analyses provide further evidence of data quality. This annotated, multilingual fMRI dataset facilitates future re-analysis that addresses cross-linguistic commonalities and differences in the neural substrate of language processing on multiple perceptual and linguistic levels.

neuroscience↗