Biology subjects

Blank, I. A.

Publications and source records attributed to Blank, I. A..

12 recordsLinked to original sources

bioRxiv2025

Illusions of Alignment Between Large Language Models and Brains Emerge From Fragile Methods and Overlooked Confounds

Emerging research seeks to draw neuroscientific insights from the neural predictivity of large language models (LLMs). However, as results continue to be generated at a rapid pace, there is a growing need for large-scale assessments of their robustness. Here, we analyze a wide range of models, methodological approaches, and neural datasets. We find that some methodological approaches, particularly the use of shuffled train-test splits, have led to many impactful yet unreliable findings, and that the method by which activations are extracted from LLMs can bias results to favor particular model classes. Moreover, we find that confounding variables, particularly positional signals and word rate, perform competitively with trained LLMs and fully account for the neural predictivity of untrained LLMs. In summary, our results suggest that theoretically interesting connections between LLMs and brains on three neural datasets are driven largely by fragile methodologies and overlooked confounds.

neuroscience↗

bioRxiv2023

Functional characterization of the language network of polyglots and hyperpolyglots with precision fMRI

How do polyglots--individuals who speak five or more languages--process their languages, and what can this population tell us about the language system? Using fMRI, we identified the language network in each of 34 polyglots (including 16 hyperpolyglots with knowledge of 10+ languages) and examined its response to the native language, non-native languages of varying proficiency, and unfamiliar languages. All language conditions engaged all areas of the language network relative to a control condition. Languages that participants rated as higher-proficiency elicited stronger responses, except for the native language, which elicited a similar or lower response than a non-native language of similar proficiency. Furthermore, unfamiliar languages that were typologically related to the participants high-to-moderate-proficiency languages elicited a stronger response than unfamiliar unrelated languages. The results suggest that the language networks response magnitude scales with the degree of engagement of linguistic computations (e.g., related to lexical access and syntactic-structure building). We also replicated a prior finding of weaker responses to native language in polyglots than non-polyglot bilinguals. These results contribute to our understanding of how multiple languages co-exist within a single brain and provide new evidence that the language network responds more strongly to stimuli that more fully engage linguistic computations.

neuroscience↗

bioRxiv2022

The language network reliably 'tracks' naturalistic meaningful non-verbal stimuli

The language network, comprised of brain regions in the left frontal and temporal cortex, responds robustly and reliably during language comprehension but shows little or no response during many non-linguistic cognitive tasks (e.g., Fedorenko & Blank, 2020). However, one domain whose relationship with language remains debated is semantics--our conceptual knowledge of the world. Given that the language network responds strongly to meaningful linguistic stimuli, could some of this response be driven by the presence of rich conceptual representations encoded in linguistic inputs? In this study, we used a naturalistic cognition paradigm to test whether the cognitive and neural resources that are responsible for language processing are also recruited for processing semantically rich non-verbal stimuli. To do so, we measured BOLD responses to a set of [~]5-minute-long video and audio clips that consisted of meaningful event sequences but did not contain any linguistic content. We then used the inter-subject correlation (ISC) approach (Hasson et al., 2004) to examine the extent to which the language network tracks these stimuli, i.e. exhibits stimulus-related variation. Across all the regions of the language network, non-verbal meaningful stimuli elicited reliable ISCs. These ISCs were higher than the ISCs elicited by semantically impoverished non-verbal stimuli (e.g., a music clip), but substantially lower than the ISCs elicited by linguistic stimuli. Our results complement earlier findings from controlled experiments (e.g., Ivanova et al., 2021) in providing further evidence that the language network shows some sensitivity to semantic content in non-verbal stimuli.

neuroscience↗

bioRxiv2021

Robust effects of working memory demand during naturalistic language comprehension in language-selective cortex

A standard view of human language processing is that comprehenders build richly structured mental representations of natural language utterances, word by word, using computationally costly memory operations supported by domain-general working memory resources. However, three core claims of this view have been questioned, with some prior work arguing that (1) rich word-by-word structure building is not a core function of the language comprehension system, (2) apparent working memory costs are underlyingly driven by word predictability (surprisal), and/or (3) language comprehension relies primarily on domain-general rather than domain-specific working memory resources. In this work, we simultaneously evaluate all three of these claims using naturalistic comprehension in fMRI. In each participant, we functionally localize (a) a language-selective network and (b) a multiple-demand network that supports working memory across domains, and we analyze the responses in these two networks of interest during naturalistic story listening with respect to a range of theory-driven predictors of working memory demand under rigorous surprisal controls. Results show robust surprisal-independent effects of word-by-word memory demand in the language network and no effect of working memory demand in the multiple demand network. Our findings thus support the view that language comprehension (1) entails word-by-word structure building using (2) computationally intensive memory operations that are not explained by surprisal. However, these results challenge (3) the domain-generality of the resources that support these operations, instead indicating that working memory operations for language comprehension are carried out by the same neural resources that store linguistic knowledge. Significance StatementThis study uses fMRI to investigate signatures of working memory (WM) demand during naturalistic story listening, using a broad range of theoretically motivated estimates of WM demand. Results support a strong effect of WM demand in language-selective brain regions but no effect of WM demand in "multiple demand" regions that have previously been associated with WM in non-linguistic domains. We further show evidence that WM effects in language regions are distinct from effects of word predictability. Our findings support a core role for WM in incremental language processing, using WM resources that are specialized for language.

neuroscience↗

bioRxiv2021

Differential tracking of linguistic vs. mental state content in naturalistic stimuli by language and Theory of Mind (ToM) brain networks

Language and social cognition, especially the ability to reason about mental states, known as Theory of Mind (ToM), are deeply related in development and everyday use. However, whether these cognitive faculties rely on distinct, overlapping, or the same mechanisms remains debated. Some evidence suggests that, by adulthood, language and ToM draw on largely distinct--though plausibly interacting--cortical networks. However, the broad topography of these networks is similar, and some have emphasized the importance of social content / communicative intent in the linguistic signal for eliciting responses in the language areas. Here, we combine the power of individual-subjects functional localization with the naturalistic-cognition inter-subject correlation approach to illuminate the language-ToM relationship. Using fMRI, we recorded neural activity as participants (n=43) listened to stories and dialogs with mental state content (+linguistic, +ToM), viewed silent animations and live action films with mental state content but no language (-linguistic, +ToM), or listened to an expository text (+linguistic, -ToM). The ToM network robustly tracked stimuli rich in mental state information regardless of whether mental states were conveyed linguistically or non-linguistically, while tracking a +linguistic/-ToM stimulus only weakly. In contrast, the language network tracked linguistic stimuli more strongly than a) non-linguistic stimuli, and than b) the ToM network, and showed reliable tracking even for the linguistic condition devoid of mental state content. These findings suggest that in spite of their indisputably close links, language and ToM dissociate robustly in their neural substrates--and thus plausibly cognitive mechanisms--including during the processing of rich naturalistic materials.

neuroscience↗

bioRxiv2020

Artificial Neural Networks Accurately Predict Language Processing in the Brain

The neuroscience of perception has recently been revolutionized with an integrative modeling approach in which computation, brain function, and behavior are linked across many datasets and many computational models. By revealing trends across models, this approach yields novel insights into cognitive and neural mechanisms in the target domain. We here present a first systematic study taking this approach to higher-level cognition: human language processing, our species signature cognitive skill. We find that the most powerful transformer models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities (fMRI, ECoG). Models neural fits ( brain score) and fits to behavioral responses are both strongly correlated with model accuracy on the next-word prediction task (but not other language tasks). Model architecture appears to substantially contribute to neural fit. These results provide computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain. SignificanceLanguage is a quintessentially human ability. Research has long probed the functional architecture of language processing in the mind and brain using diverse brain imaging, behavioral, and computational modeling approaches. However, adequate neurally mechanistic accounts of how meaning might be extracted from language are sorely lacking. Here, we report an important first step toward addressing this gap by connecting recent artificial neural networks from machine learning to human recordings during language processing. We find that the most powerful models predict neural and behavioral responses across different datasets up to noise levels. Models that perform better at predicting the next word in a sequence also better predict brain measurements - providing computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain.

neuroscience↗

bioRxiv2020

Incremental language comprehension difficulty predicts activity in the language network but not the multiple demand network

What role do domain-general executive functions play in human language comprehension? To address this question, we examine the relationship between behavioral measures of comprehension and neural activity in the domain-general "multiple demand" (MD) network, which has been linked to constructs like attention, working memory, inhibitory control, and selection, and implicated in diverse goal-directed behaviors. Specifically, fMRI data collected during naturalistic story listening are compared to theory-neutral measures of online comprehension difficulty and incremental processing load (reading times and eye-fixation durations). Critically, to ensure that variance in these measures is driven by features of the linguistic stimulus rather than reflecting participant-or trial-level variability, the neuroimaging and behavioral datasets were collected in non-overlapping samples. We find no behavioral-neural link in functionally localized MD regions; instead, this link is found in the domain-specific, fronto-temporal "core language network", in both left hemispheric areas and their right hemispheric homologues. These results argue against strong involvement of domain-general executive circuits in language comprehension.

neuroscience↗

bioRxiv2019

The domain-general multiple demand (MD) network does not support core aspects of language comprehension: a large-scale fMRI investigation

Aside from the language-selective left-lateralized fronto-temporal network, language comprehension sometimes additionally recruits a domain-general bilateral fronto-parietal network implicated in executive functions: the multiple demand (MD) network. However, the nature of the MD networks contributions to language comprehension remains debated. To illuminate the role of this network in language processing, we conducted a large-scale fMRI investigation using data from 30 diverse word and sentence comprehension experiments (481 unique participants, 678 scanning sessions). In line with prior findings, the MD network was active during many language tasks. Moreover, similar to the language-selective network, which is robustly lateralized to the left hemisphere, these responses were stronger in the left-hemisphere MD regions. However, in stark contrast with the language-selective network, the MD network responded more strongly (i) to lists of unconnected words than to sentences, and critically, (ii) in paradigms with an explicit task compared to passive comprehension paradigms. In fact, many passive comprehension tasks failed to elicit a response above the fixation baseline in the MD network, in contrast to strong responses in the language-selective network. In tandem, these results argue against a role for the MD network in core aspects of sentence comprehension like inhibiting irrelevant meanings or parses, keeping intermediate representations active in working memory, or predicting upcoming words or structures. These results align with recent evidence of relatively poor tracking of the linguistic signal by the MD regions during naturalistic comprehension, and instead suggest that the MD networks engagement during language processing likely reflects effort associated with extraneous task demands. Significance StatementDomain-general executive processes, like working memory and cognitive control, have long been implicated in language comprehension, including in neuroimaging studies that have reported activation in domain-general multiple demand (MD) regions for linguistic manipulations. However, much prior evidence has come from paradigms where language interpretation is accompanied by extraneous tasks. Using a large fMRI dataset (30 experiments/481 participants/678 sessions), we demonstrate that MD regions are engaged during language comprehension in the presence of task demands, but not during passive reading/listening--conditions that strongly activate the fronto-temporal language network. These results present a fundamental challenge to proposals whereby linguistic computations, like inhibiting irrelevant meanings, keeping representations active in working memory, or predicting upcoming elements, draw on domain-general executive resources.

neuroscience↗

bioRxiv2019

fMRI reveals language-specific predictive coding during naturalistic sentence comprehension

Much research in cognitive neuroscience supports prediction as a canonical computation of cognition across domains. Is such predictive coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that predictive coding in the brains response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed continuous-time deconvolutional regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found effects of prediction measures in the language network but not in the domain-general multiple-demand network, which supports executive control processes and has been previously implicated in language comprehension. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.

neuroscience↗

bioRxiv2019

The small and efficient language network of polyglots and hyper-polyglots

Acquiring a foreign language is challenging for many adults. Yet certain individuals choose to acquire sometimes dozens of languages, and often just for fun. Is there something special about the minds and brains of such polyglots? Using robust individual-level markers of language activity, measured with fMRI, we compared native language processing in polyglots versus matched controls. Polyglots (n=17, including 9 \"hyper-polyglots\" with proficiency in 10-55 languages) used fewer neural resources to process language: their activations were smaller in both magnitude and extent. This difference was spatially and functionally selective: the groups were similar in their activation of two other brain networks - the multiple demand network and the default mode network. We hypothesize that the activation reduction in the language network is experientially driven, such that the acquisition and use of multiple languages makes language processing generally more efficient. However, genetic and longitudinal studies will be critical to distinguish this hypothesis from the one whereby polyglots brains already differ at birth or early in development. This initial characterization of polyglots language network opens the door to future investigations of the cognitive and neural architecture of individuals who gain mastery of multiple languages, including changes in this architecture with linguistic experiences.

neuroscience↗

bioRxiv2019

No evidence for functional distinctions across fronto-temporal language regions in their temporal receptive windows

The \"core language network\" consists of left temporal and frontal regions that are selectively engaged in linguistic processing. Whereas the functional differences across these regions have long been debated, many accounts propose distinctions in terms of representational grain-size--e.g., words vs. sentences--or processing time-scale, i.e., operating on local linguistic features vs. larger spans of input. Indeed, the topography of language regions appears to overlap with a cortical hierarchy reported by Lerner et al. (2011) wherein mid-posterior temporal regions are sensitive to low-level features of speech, surrounding areas--to word-level information, and inferior frontal areas--to sentence-level information and beyond. However, the correspondence between the language network and this hierarchy of \"temporal receptive windows\" (TRWs) is difficult to establish because the precise anatomical locations of language regions vary across individuals. To directly test this correspondence, we first identified language regions in each participant with a task-based localizer, which confers high functional resolution to the study of TRWs (traditionally based on stereotactic coordinates); then, we characterized regional TRWs with the naturalistic story listening paradigm of Lerner et al. (2011), which augments task-based characterizations of the language network by more closely resembling comprehension \"in the wild\". We find no region-by-TRW interactions across temporal and inferior frontal regions, which are all sensitive to both word-level and sentence-level information. Therefore, the language network as a whole constitutes a unique stage of information integration within a broader cortical hierarchy.\n\nHighlightsO_LILanguage regions are identified with task-based, participant-specific localization.\nC_LIO_LIA progressively scrambled naturalistic story probes regional processing timescales.\nC_LIO_LIWidespread sensitivity to scrambling at the timescales of both words and sentences.\nC_LIO_LINo processing timescale distinctions across temporal and inferior-frontal regions.\nC_LIO_LIThese regions all occupy a common, unique stage in a broader processing hierarchy.\nC_LI

neuroscience↗

bioRxiv2017

Multiple Demand (MD) system’s activity predicts individual differences in working memory and fluid intelligence

Numerous brain lesion and fMRI studies have linked individual differences in executive abilities and fluid intelligence to brain regions of the fronto-parietal "multiple-demand" (MD) network. Yet, fMRI studies have yielded conflicting evidence as to whether better executive abilities are associated with stronger or weaker MD activations and whether this relationship is restricted to the MD network. Here, in a large-sample (n=216) fMRI investigation, we found that stronger activity in MD regions - functionally defined in individual participants - was robustly associated with more accurate and faster responses on a spatial working memory task performed in the scanner, as well as fluid intelligence measured independently (n=114). In line with some prior claims about a relationship between language and fluid intelligence, we also found a weak association between activity in the brain regions of the left fronto-temporal language network during an independent passive reading task, and performance on the working memory task. However, controlling for the level of MD activity abolished this relationship, whereas the MD activity-behavior association remained highly reliable after controlling for the level of activity in the language network. Finally, we demonstrate how unreliable MD activity measures, coupled with small sample sizes, could falsely lead to the opposite, negative, association that has been reported in some prior studies. Taken together, these results demonstrate that a core component of individual differences variance in executive abilities and fluid intelligence is selectively and robustly positively associated with the level of activity in the MD network, a result that aligns well with lesion studies.

neuroscience↗