bioRxiv Science⌕ Search

Biology subjects

Chavez, A. G. L.

Publications and source records attributed to Chavez, A. G. L..

2 recordsLinked to original sources

Polysemanticity in human hippocampal neurons

To comprehend language, the brain must navigate a high-dimensional semantic landscape while seamlessly contextualizing meaning. Inspired by recent advances in the mechanistic interpretability of large language models (LLMs), we hypothesized that the brain utilizes polysemanticity, a coding strategy wherein individual neurons represent multiple semantically unrelated features through high-dimensional superposition (Elhage et al., 2022; Olah et al., 2020). We recorded single-unit activity from the human hippocampus during podcast listening. We found that hippocampal neurons exhibit dense semantic codes characterized by multiple tuning peaks with an overdispersed, isotropic geometry. This geometry satisfies the theoretical requirements for interference minimization in superimposed codes. Furthermore, semantic responses are strongly modulated by lexical and speaker-identity context; nonetheless, the underlying population geometry remains stable. This coding strategy permits rapid contextualization without requiring specialized, context-specific neurons. Indeed, we show clear pattern separation of similar terms, along with pattern completion for held-out words. Together, these results demonstrate that the human brain leverages superposition to solve a universal computational problem: maximizing semantic capacity within a constrained representational space.

neuroscience↗

Semantic axes in the brain support analogical representations

ABSTRACTIn language models of word meaning, directions in the embedding space often correspond to semantic features that can be reused across different words. For example, a single direction corresponding to gender may differentiate word pairs like "boy/girl", "uncle/aunt" and "king/queen". Here we show that the same principle governs semantically driven neural responses in the human brain. We recorded populations of single neurons during podcast listening and identified word sets with consistent meaning differences. Across fifteen sets, including gender, plural, and negation, we observed consistent vectorial directions, resulting in parallelogram structures within the neural manifold. Deviation from parallelism in large language models (LLMs) predicted corresponding deviations in brain-derived parallelism. Among pronouns, vectors corresponding to case, number and person exhibited parallelogram structures individually and, collectively, obeyed the principle of commutativity, resulting in a prismatic structure. Finally, different semantic variables were preferentially associated with discrete groups of neurons, consistent with energy-efficiency theories. Together, these results establish a geometric foundation for the neural encoding of word meaning.

neuroscience↗