Biology subjects

Kauf, C.

Publications and source records attributed to Kauf, C..

4 recordsLinked to original sources

bioRxiv2025

Semantic reasoning takes place largely outside the language network

The brains language network is often implicated in the representation and manipulation of abstract semantic knowledge. However, this view is inconsistent with a large body of evidence suggesting that language processing is neurally distinct from the rest of cognition. Here, we use precision brain imaging to uncover a set of brain regions, separate from the language network, that are engaged in semantic processing of both linguistic and pictorial stimuli. In three fMRI experiments, participants (total n=41 tested across 49 sessions) viewed sentences and pictures depicting simple events. In separate blocks, they performed either a semantic task or a difficulty-matched perceptual task. Across all three experiments, several areas in left lateral prefrontal cortex, left temporo-parietal cortex, and right cerebellum responded to semantic tasks for both sentences and pictures. These semantic processing areas are spatially and functionally distinct from the nearby language-selective areas, as well as from the multiple demand and default mode networks, exhibiting a unique response profile. Our results provide evidence for a new kind of selectivity in the human brain and pave the way for future explorations of the neural mechanisms that underlie semantic reasoning.

neuroscience↗

bioRxiv2024

Linguistic inputs must be syntactically parsable to fully engage the language network

Human language comprehension is remarkably robust to ill-formed inputs (e.g., word transpositions). This robustness has led some to argue that syntactic parsing is largely an illusion, and that incremental comprehension is more heuristic, shallow, and semantics-based than is often assumed. However, the available data are also consistent with the possibility that humans always perform rule-like symbolic parsing and simply deploy error correction mechanisms to reconstruct ill-formed inputs when needed. We put these hypotheses to a new stringent test by examining brain responses to a) stimuli that should pose a challenge for syntactic reconstruction but allow for complex meanings to be built within local contexts through associative/shallow processing (sentences presented in a backward word order), and b) grammatically well-formed but semantically implausible sentences that should impede semantics-based heuristic processing. Using a novel behavioral syntactic reconstruction paradigm, we demonstrate that backward- presented sentences indeed impede the recovery of grammatical structure during incremental comprehension. Critically, these backward-presented stimuli elicit a relatively low response in the language areas, as measured with fMRI. In contrast, semantically implausible but grammatically well-formed sentences elicit a response in the language areas similar in magnitude to naturalistic (plausible) sentences. In other words, the ability to build syntactic structures during incremental language processing is both necessary and sufficient to fully engage the language network. Taken together, these results provide strongest to date support for a generalized reliance of human language comprehension on syntactic parsing. Significance statementWhether language comprehension relies predominantly on structural (syntactic) cues or meaning- related (semantic) cues remains debated. We shed new light on this question by examining the language brain areas responses to stimuli where syntactic and semantic cues are pitted against each other, using fMRI. We find that the language areas respond weakly to stimuli that allow for local semantic composition but cannot be parsed syntactically--as confirmed in a novel behavioral paradigm--and they respond strongly to grammatical but semantically implausible sentences, like the famous Colorless green ideas sleep furiously sentence. These findings challenge accounts of language processing that suggest that syntactic parsing can be foregone in favor of shallow semantic processing.

neuroscience↗

bioRxiv2023

Lexical semantic content, not syntactic structure, is the main contributor to ANN-brain similarity of fMRI responses in the language network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI dataset of responses to n=627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we i) perturbed sentences word order, ii) removed different subsets of words, or iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical semantic content of the sentence (largely carried by content words) rather than the sentences syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANNs embedding space and decrease the ANNs ability to predict upcoming tokens in those stimuli. Further, results are robust to whether the mapping model is trained on intact or perturbed stimuli, and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result--that lexical- semantic content is the main contributor to the similarity between ANN representations and neural ones--aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

neuroscience↗

bioRxiv2020

Artificial Neural Networks Accurately Predict Language Processing in the Brain

The neuroscience of perception has recently been revolutionized with an integrative modeling approach in which computation, brain function, and behavior are linked across many datasets and many computational models. By revealing trends across models, this approach yields novel insights into cognitive and neural mechanisms in the target domain. We here present a first systematic study taking this approach to higher-level cognition: human language processing, our species signature cognitive skill. We find that the most powerful transformer models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities (fMRI, ECoG). Models neural fits ( brain score) and fits to behavioral responses are both strongly correlated with model accuracy on the next-word prediction task (but not other language tasks). Model architecture appears to substantially contribute to neural fit. These results provide computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain. SignificanceLanguage is a quintessentially human ability. Research has long probed the functional architecture of language processing in the mind and brain using diverse brain imaging, behavioral, and computational modeling approaches. However, adequate neurally mechanistic accounts of how meaning might be extracted from language are sorely lacking. Here, we report an important first step toward addressing this gap by connecting recent artificial neural networks from machine learning to human recordings during language processing. We find that the most powerful models predict neural and behavioral responses across different datasets up to noise levels. Models that perform better at predicting the next word in a sequence also better predict brain measurements - providing computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain.

neuroscience↗