bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.64898/2026.07.12.738026

Computational Counterfactuals Reveal Non-Additive Audiovisual Semantics in Natural Movie Responses

Li, M.
View original source ↗Export metadata ↓

Abstract

Natural audiovisual perception may not be fully captured by decomposing movies into auditory and visual streams. I introduce a computational-counterfactual framework that keeps movie viewing intact while varying only AI-derived descriptions of the same clips. Using 7 Tesla movie fMRI imaging data from 176 participants, I tested whether cortical responses were better predicted by native audiovisual semantics than by a dimension-matched additive reconstruction from audio-only and video-only descriptions. The native model outperformed the matched additive baseline under content-aware purged cross-validation, with strongest gains in auditory, visual, and dorsal attention systems. Representational-similarity, feature-replacement, and content-gating analyses showed that the advantage reflected feature- and network-specific routing linked to coherent audiovisual semantic emergence rather than raw auditory-visual discrepancy. The effect survived stronger temporal purging and repeat-content exclusion, suggesting that intact movie viewing evokes cortical structure aligned with native audiovisual meaning beyond additive unimodal semantics.

Explore related subjects

neurosciencePreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Li, M.. 2026-07-17. Computational Counterfactuals Reveal Non-Additive Audiovisual Semantics in Natural Movie Responses. https://doi.org/10.64898/2026.07.12.738026

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Neurodegeneration-inducing macromolecules exit the brain via nanovascular conduits formed by reticular fibroblasts

Accumulation of proteins such as amyloid beta (Abeta), hyperphosphorylated tau and alpha-synuclein within the brain alters neural information processing and causes neurodegeneration(1-3), but how toxic solutes are cleared from the brain remains highly controversial(4,5). Proposed exit routes include efflux across endothelial cells into the blood(6,7), and movement to the pial surface via vasomotion-induced pumping along spaces within arteriolar smooth muscle(8) or via outflow along the perivascular space of ascending venules promoted by water flux through astrocytes (the glymphatic system(9)). From the pial surface of the brain, drainage may continue to dural lymphatics, along the outer sheaths of exiting cranial nerves and across the cribriform plate(10-14). We now report the presence, in mice and humans, of 2 micron diameter conduits that remove fluorescently labelled tau and Abeta from the brain. These conduits form a spatially-organised mesh within the walls of penetrating arterioles and pial arteries, and around the surface of ascending venules and deep cerebral and pial veins. They course through the pial and arachnoid layers to span the CSF space, wrapping the brain and cranial nerves. They are formed of reticular fibroblasts, which label for VE-cadherin(15) and PDGFRalpha(16), the lymphatic markers(17) podoplanin, VEGFR3 and Prox1, and reticular fibroblast extracellular matrix components collagen I and VI(16,18-20). Parenchymal tau drains from the brain at a similar rate via arteriolar conduits and via conduits around venules, arguing against preferential removal by a glymphatic mechanism. In Alzheimer's disease model mice, Abeta is seen traversing these lymph node-like conduits. Modulation of molecular transfer via this route may accelerate or delay cognitive decline, and slowed transfer from arteriolar to pial-arachnoid conduits may initiate cerebral amyloid angiopathy.

neuroscience↗
bioRxiv2026

Analysis of the influence of gradual changes in matrix sentence similarity on neural envelope tracking

Neural tracking of speech is a well-established phenomenon in neuroscience. However, for speech signals with a fixed structure, significant correlations between speech envelopes and neurophysiological representations occur even for unheard sentences. We exploit a structured speech-in-noise matrix hearing test (Oldenburger Sentence Test, OLSA) to systematically quantify the relationship between acoustic sentence similarity and neural tracking. Simultaneous magnetoencephalography (MEG) and 76-channel electroencephalography (EEG) data, including 16 channels positioned directly around the ears (ear-EEG), were recorded from 21 young adults with normal hearing during the presentation of clean-speech audiobooks and OLSA sentences at six signal-to-noise ratios. A linear decoder trained on audiobooks reconstructed OLSA sentence envelopes. Reconstruction accuracies were compared using a linear mixed model across heard (matched) and unheard (mismatched) sentences of varying acoustic similarity. Significant reconstruction accuracies were achieved across MEG, EEG, and ear-EEG for both matched and mismatched sentences. For mismatched sentences, these accuracies gradually increased with their acoustic similarity to the heard speech data. The high similarity between sentences, which is especially prominent in matrix tests, can cause significant spurious tracking for mismatched stimuli. This effect can reach levels comparable to those of matched sentences and can be mistaken for true neural tracking. Robust neural tracking across modalities further supported the established viability of ear-EEG compared to whole-head systems.

neuroscience↗
bioRxiv2026

Seizures and tauopathy following neurotrauma are mediated by prion protein and metabotropic glutamate receptor 5

Traumatic brain injury (TBI) is one of the world's leading causes of death and disability and a major risk factor for dementias. The primary dementia associated with TBI is chronic traumatic encephalopathy (CTE), a neurodegenerative disease classified as a tauopathy, in which toxic tau molecules lead to disease pathologies and degeneration. The processes that lead to tauopathy and subsequent dementia after TBI remain unclear. Here, we built upon the finding that seizures after TBI may be a mechanism leading to tauopathy, by dissecting the functions of the metabotropic glutamate receptor 5 - cellular prion protein (mGluR5-PrPC) pathway. We delivered TBI to larval in a blast paradigm, and quantified aggregation of Tau via a genetically-encoded Tau-GFP fusion reporter. Zebrafish larvae lacking prp2 (homolog of mammalian cellular Prion Protein, PrPC) displayed a 168% increase in post-traumatic seizures activity after TBI. An mGluR5 agonist (CHPG) reduced post-traumatic seizures, whereas an mGluR5 antagonist (MPEP) increased post-traumatic seizures. Moreover, agonizing mGluR5 reduced tau aggregation and antagonizing mGluR5 increased tau burden. Larvae seizing from convulsants, rather than TBI, were treated with CHPG/MPEP and provided a similar pattern of outcomes, suggesting seizures may be a factor needed for mGluR5 activity to influence tau aggregation. The PrPC-mGluR5 pathway is proposed as one candidate pathomechanism linking TBI to subsequent seizures and tauopathy, and thus it warrants investigation as a target for prophylactic interventions.

neuroscience↗