Biology subjects

Desbordes, T.

Publications and source records attributed to Desbordes, T..

3 recordsLinked to original sources

bioRxiv2026

From Images and Words to Modality-Independent Semantic Representations: When and Where They Emerge

Conceptual knowledge can be accessed from different modalities, such as seeing an image or reading a written word. While prior neuroimaging studies implicate anterior temporal cortex in modality-independent semantic representations, when and how representations evoked by images and words converge onto a modality-independent code remains unclear. Moreover, evidence for modality-independent category representations comes largely from tasks that explicitly required semantic processing, leaving open whether such representations emerge when semantic category and stimulus modality are task-irrelevant. We magnetoencephalography (MEG) to determine when and where modality-independent semantic category representations emerge across images and words when participants (n = 32) encoded six-item sequences containing an exemplar of six semantic categories counterbalanced across positions and reported the ordinal position of a probed item. Decoding performance was above chance for all categories in each modality from approximately 70 ms relative to item presentation. Generalization peaked at similar latencies for the two modalities (~520-530 ms), where image and word classifiers had the most similar coefficients. In both transfer directions, activity elicited by images generalized mostly to later activity elicited by words, and a given period of the image response shared its category code with a longer period of the word response. Source-space decoding around the peak of generalization identified the same regions for both transfer directions: left and right anterior temporal lobe, perisylvian cortex (more extensive in the left hemisphere), right visual cortex, and bilateral orbitofrontal cortex, indicating a modality-independent category code distributed across the cortex.

neuroscience↗

bioRxiv2025

Temporal evolution of Neural Codes: The Added Value of a Geometric Approach to Linear Coefficients

Multivariate decoding analyses have become a cornerstone method in cognitive neuroscience. When applied to time-resolved brain imaging signals, they provide insights into the temporal dynamics of information processing in the brain. In particular, the temporal generalization (TG) method--where a decoder trained at one time point is tested on others--is commonly used to assess the stability of neural representations over time. However, TG performance can be ambiguous: distinct representational dynamics--such as sparse versus distributed activity, or scaling of activity versus recruitment of new units--can yield similar TG matrices. Moreover, even when generalization is strong, underlying neural representations may still be evolving in ways that TG alone fails to reveal. This ambiguity of performance profiles can mask meaningful changes in the geometry of neural representations. In this study, we use controlled simulations to demonstrate how different dynamic processes can produce indistinguishable TG profiles. To resolve these ambiguities, we propose a complementary approach based on the geometry of the learned linear coefficients. Specifically, we quantify the Rotation Angle {theta} between decision subspaces (with cosine similarity) and the Feature Density (capturing whether feature contributions are distributed or sparse). Together, these measures complement TG analyses, revealing how neural representations evolve in space and time. Beyond time-resolved decoding, our approach applies broadly to any linear model, offering a geometric perspective on representational dynamics. Graphical abstract O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=113 SRC="FIGDIR/small/681582v1_ufig1.gif" ALT="Figure 1"> View larger version (19K): org.highwire.dtl.DTLVardef@c1584forg.highwire.dtl.DTLVardef@18cdd30org.highwire.dtl.DTLVardef@10d7c22org.highwire.dtl.DTLVardef@50a6fa_HPS_FORMAT_FIGEXP M_FIG C_FIG

neuroscience↗

bioRxiv2023

Dimensionality and ramping: Signatures of sentence integration in the dynamics of brains and deep language models

A sentence is more than the sum of its words: its meaning depends on how they combine with one another. The brain mechanisms underlying such semantic composition remain poorly understood. To shed light on the neural vector code underlying semantic composition, we introduce two hypotheses: First, the intrinsic dimensionality of the space of neural representations should increase as a sentence unfolds, paralleling the growing complexity of its semantic representation, and second, this progressive integration should be reflected in ramping and sentence-final signals. To test these predictions, we designed a dataset of closely matched normal and Jabberwocky sentences (composed of meaningless pseudo words) and displayed them to deep language models and to 11 human participants (5 men and 6 women) monitored with simultaneous magneto-encephalography and intracranial electro-encephalography. In both deep language models and electrophysiological data, we found that representational dimensionality was higher for meaningful sentences than Jabberwocky. Furthermore, multivariate decoding of normal versus Jabberwocky confirmed three dynamic patterns: (i) a phasic pattern following each word, peaking in temporal and parietal areas, (ii) a ramping pattern, characteristic of bilateral inferior and middle frontal gyri, and (iii) a sentence-final pattern in left superior frontal gyrus and right orbitofrontal cortex. These results provide a first glimpse into the neural geometry of semantic integration and constrain the search for a neural code of linguistic composition. Significance statementStarting from general linguistic concepts, we make two sets of predictions in neural signals evoked by reading multi-word sentences. First, the intrinsic dimensionality of the representation should grow with additional meaningful words. Second, the neural dynamics should exhibit signatures of encoding, maintaining, and resolving semantic composition. We successfully validated these hypotheses in deep Neural Language Models, artificial neural networks trained on text and performing very well on many Natural Language Processing tasks. Then, using a unique combination of magnetoencephalography and intracranial electrodes, we recorded high-resolution brain data from human participants while they read a controlled set of sentences. Time-resolved dimensionality analysis showed increasing dimensionality with meaning, and multivariate decoding allowed us to isolate the three dynamical patterns we had hypothesized.

neuroscience↗