Biology subjects

Deniz, F.

Publications and source records attributed to Deniz, F..

9 recordsLinked to original sources

bioRxiv2026

Representations of semantic relations in the human cerebral cortex

An essential aspect of human cognition is the ability to explicitly think about semantic relations between concepts. Neuroimaging studies have found that individual concepts are encoded by distributed patterns of cortical activity, but relatively little is known about how semantic relations between concepts are encoded in the brain. Some theoretical models suggest that relation representations are embedded within concept representations, while others suggest that relation representations are independent of any specific concept pair. We designed a study to compare how semantic relations and concepts are encoded across the cerebral cortex. To characterize how relations are encoded across cortex, fMRI was used to record brain activity while six participants each answered over one thousand questions about different semantic relations. We find that relations are encoded independently of the specific concepts that are connected in any particular instance of the relation. Our results further suggest that relations and concepts are represented in the same set of cortical regions, and that, within these regions, each location is preferentially selective for specific relations. Overall, these results suggest that in the human cerebral cortex, relations and concepts may have the same type of functional representation.

neuroscience↗

bioRxiv2025

Concreteness shapes semantic representations in bilingual brains

Behavioral studies show that humans process concrete words more similarly across languages than abstract words. This suggests that in bilingual brains, semantic representations may be more similar across languages for concrete concepts than for abstract concepts, but existing neuroimaging evidence is inconclusive. Here, we analyzed functional magnetic resonance imaging (fMRI) data from fluent Chinese-English bilinguals who read several hours of naturalistic narratives in both languages. We used encoding models to estimate voxelwise tuning towards concrete and abstract concepts in each language separately. We then quantified the similarity of cortical semantic representations across languages. First, we find that the cortical organization of concreteness tuning is consistent across languages. Second, semantic representations are similar across languages for both concrete-tuned voxels and abstract-tuned voxels. Third, we find that for abstract concepts, cross-language similarity of semantic representations may be driven by emotionality. Overall, these findings reveal how concreteness affects semantic representations in bilingual brains.

neuroscience↗

bioRxiv2025

Optimizing Language Model Embeddings to Voxel Activity Improves Brain Activity Predictions

Recent studies have shown that contextual semantic embeddings from language models can accurately predict human brain activity during language processing. However, most studies use contextual embeddings with the same context length and model layer for all voxels, potentially overlooking meaningful variations across the brain. In this study, we investigate whether optimizing contextual embeddings for individual voxels improves their ability to predict brain activity during reading. We optimize embeddings for each voxel by selecting the best-predicting context length, model layer, or both. We perform this optimization with two different types of stimuli (isolated sentences and narratives), and quantify the performance gains of optimized embeddings over standard fixed embeddings. Our results show that voxel-specific optimization substantially improves the prediction accuracy of contextual semantic embeddings. These findings demonstrate that voxel-specific contextual tuning provides a more accurate and nuanced account of how the contextual semantic information is represented across the cortex.

neuroscience↗

bioRxiv2025

Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models

Recent studies have demonstrated that fine-tuning language models with brain data can improve their semantic understanding, although these findings have so far been limited to English. Interestingly, similar to the shared multilingual embedding space of pretrained multilingual language models, human studies provide strong evidence for a shared semantic system in bilingual individuals. Here, we investigate whether fine-tuning language models with bilingual brain data changes model representations in a way that improves them across multiple languages. To test this, we fine-tune monolingual and multilingual language models using brain activity recorded while bilingual participants read stories in English and Chinese. We then evaluate how well these representations generalize to the bilingual participants first language, their second language, and several other languages that the participants are not fluent in. We assess the fine-tuned language models on brain encoding performance and downstream NLP tasks. Our results show that bilingual brain-informed fine-tuned language models outperform their vanilla (pretrained) counterparts in both brain encoding performance and most downstream NLP tasks across multiple languages. These findings suggest that brain-informed fine-tuning improves multilingual understanding in language models, offering a bridge between cognitive neuroscience and NLP research. We make our code publicly available. 2

neuroscience↗

bioRxiv2024

Do transformers and CNNs learn different concepts of brain age?

"Predicted brain age" refers to a biomarker of structural brain health derived from machine learning analysis of T1-weighted brain magnetic resonance (MR) images. A range of machine learning methods have been used to predict brain age, with convolutional neural networks (CNNs) currently yielding state-of-the-art accuracies. Recent advances in deep learning have introduced transformers, which are conceptually distinct from CNNs, and appear to set new benchmarks in various domains of computer vision. However, transformers have not yet been applied to brain age prediction. Thus, we address two research questions: First, are transformers superior to CNNs in predicting brain age? Second, do conceptually different deep learning model architectures learn similar or different "concepts of brain age"? We adapted a Simple Vision Transformer (sViT) and a Shifted Window Transformer (SwinT) to predict brain age, and compared both models with a ResNet50 on 46,381 T1-weighted structural MR images from the UK Biobank. We found that SwinT and ResNet performed on par, while additional training samples will most likely give SwinT the edge in prediction accuracy. We identified that different model architectures may characterize different (sub-)sets of brain aging effects, representing diverging concepts of brain age. Thus, we systematically tested whether sViT, SwinT and ResNet focus on different concepts of brain age by examining variations in their predictions and clinical utility for indicating deviations in neurological and psychiatric disorders. Reassuringly, we did not find substantial differences in the structure of brain age predictions between model architectures. Based on our results, the choice of deep learning model architecture does not appear to have a confounding effect on brain age studies.

neuroscience↗

bioRxiv2024

Bilingual language processing relies on shared semantic representations that are modulated by each language

Billions of people throughout the world are bilingual, and they can extract meaning from multiple languages. While some evidence suggests that there is a shared system in the human brain for processing semantic information from native and non-native languages, other evidence suggests that semantic processing is language-specific. We conducted a study to determine how semantic information for different languages is represented in the brains of bilinguals. Functional magnetic resonance imaging (fMRI) was used to record brain responses while participants read several hours of natural narratives in their native (Chinese) and non-native (English) languages. These data were then used to compare semantic representations between the two languages. We find that semantic representations are largely shared between languages, but that there are fine-grained differences in the representation of some semantic categories across languages. These results reconcile current competing theories of bilingual language processing. Significance StatementBilinguals understand the meaning of words in multiple languages. Whether this capacity reflects a shared brain system for processing both native and non-native languages, or whether processing is language-specific is still unclear. Here, we examine whether and how semantic representations in the brain support shared and/or language-specific processing. We recorded brain activity from participants reading narratives in their native (Chinese) and non-native (English) languages, and modeled how their brains represent word meaning in each language. We show that semantic representations are similar between the two language conditions, and that these representations are systematically modulated between native and non-native language comprehension.

neuroscience↗

bioRxiv2023

The Cortical Representation of Language Timescales is Shared between Reading and Listening

Language comprehension involves integrating low-level sensory inputs into a hierarchy of increasingly high-level features. Prior work studied brain representations of different levels of the language hierarchy, but has not determined whether these brain representations are shared between written and spoken language. To address this issue, we analyzed fMRI BOLD data recorded while participants read and listened to the same narratives in each modality. Levels of the language hierarchy were operationalized as timescales, where each timescale refers to a set of spectral components of a language stimulus. Voxelwise encoding models were used to determine where different timescales are represented across the cerebral cortex, for each modality separately. These models reveal that between the two modalities timescale representations are organized similarly across the cortical surface. Our results suggest that, after low-level sensory processing, language integration proceeds similarly regardless of stimulus modality.

neuroscience↗

bioRxiv2022

Attention weights accurately predict language representations in the brain

In Transformer-based language models (LMs) the attention mechanism converts token embeddings into contextual embeddings that incorporate information from neighboring words. The resulting contextual hidden state embeddings have enabled highly accurate models of brain responses, suggesting that the attention mechanism constructs contextual embeddings that carry information reflected in language-related brain representations. However, it is unclear whether the attention weights that are used to integrate information across words are themselves related to language representations in the brain. To address this question we analyzed functional magnetic resonance imaging (fMRI) recordings of participants reading English language narratives. We provided the narrative text as input to two LMs (BERT and GPT-2) and extracted their corresponding attention weights. We then used encoding models to determine how well attention weights can predict recorded brain responses. We find that attention weights accurately predict brain responses in much of the frontal and temporal cortices. Our results suggest that the attention mechanism itself carries information that is reflected in brain representations. Moreover, these results indicate cortical areas in which context integration may occur.

neuroscience↗

bioRxiv2021

Semantic representations during language comprehension are affected by context

The meaning of words in natural language depends crucially on context. However, most neuroimaging studies of word meaning use isolated words and isolated sentences with little context. Because the brain may process natural language differently from how it processes simplified stimuli, there is a pressing need to determine whether prior results on word meaning generalize to natural language. fMRI was used to record human brain activity while four subjects (two female) read words in four conditions that vary in context: narratives, isolated sentences, blocks of semantically similar words, and isolated words. We then compared the signal-to-noise ratio (SNR) of evoked brain responses, and we used a voxelwise encoding modeling approach to compare the representation of semantic information across the four conditions. We find four consistent effects of varying context. First, stimuli with more context evoke brain responses with higher SNR across bilateral visual, temporal, parietal, and prefrontal cortices compared to stimuli with little context. Second, increasing context increases the representation of semantic information across bilateral temporal, parietal, and prefrontal cortices at the group level. In individual subjects, only natural language stimuli consistently evoke widespread representation of semantic information. Third, context affects voxel semantic tuning. Finally, models estimated using stimuli with little context do not generalize well to natural language. These results show that context has large effects on the quality of neuroimaging data and on the representation of meaning in the brain. Thus, neuroimaging studies that use stimuli with little context may not generalize well to the natural regime. Significance StatementContext is an important part of understanding the meaning of natural language, but most neuroimaging studies of meaning use isolated words and isolated sentences with little context. Here we examined whether the results of neuroimaging studies that use out-of-context stimuli generalize to natural language. We find that increasing context improves the quality of neuroimaging data and changes where and how semantic information is represented in the brain. These results suggest that findings from studies using out-of-context stimuli may not generalize to natural language used in daily life.

neuroscience↗