Biology subjects

Heilbron, M.

Publications and source records attributed to Heilbron, M..

11 recordsLinked to original sources

bioRxiv2026

Generative and discriminative recurrence employ opposing strategies for robust vision

Recurrence is thought to enhance the robustness of biological vision, but how it achieves this feat is largely unknown. Perceptual robustness can be implemented through either lateral connections supporting local integration within a processing stage or feedback connections drawing on broader context from higher stages, and through either a discriminative objective optimising task-relevant classification or a generative objective learning to reconstruct the causes of visual input. But do these different types of recurrence engage distinct computational strategies? As this question is difficult to test in vivo, we endowed convolutional neural networks with varying recurrent architectures and training objectives, and evaluated the consequences for internal representations and behaviour across noise levels. Two distinct computational strategies emerged. Generative feedback followed a reductionist strategy, with representations becoming lower-dimensional through denoising, achieving robustness at moderate noise levels without noise training. Both discriminative lateral and feedback recurrence followed an expansionist strategy, increasing dimensionality to sharpen discriminability without denoising, but requiring noise training to achieve robustness. These dissociable signatures reflect fundamentally different computational mechanisms of robust vision and provide testable predictions for which form of recurrence the brain employs.

neuroscience↗

bioRxiv2026

A hierarchy of spatial predictions across human visual cortex during natural vision

Prediction is foundational to theories of perceptual processing and learning, in both neuroscience and AI. However, it remains unclear whether prediction occurs routinely during naturalistic perception, and at what level of abstraction the brain predicts. Here, we address both questions by analysing 7T fMRI recordings of humans viewing 73,000 natural images. We use deep generative models to quantify spatial predictability at multiple levels of abstraction, and relate these to retinotopically precise responses across V1-V4, while rigorously controlling for local image features. This reveals that, even during natural scene viewing, responses throughout visual cortex are modulated by spatial predictability, with more predictable inputs evoking weaker responses. In central vision, we observe a hierarchy of predictions that parallels the feature-encoding gradient: V1 is most sensitive to low-level unpredictability, with later areas progressively sensitive to higher-level unpredictability - diverging from recent proposals, in both neuroscience and AI, that prediction operates primarily at higher levels of abstraction. At higher eccentricities, prediction effects are amplified but even V1 is tuned to high-level predictability, consistent with these prior accounts. Together, these results suggest that the visual system implements distinct prediction regimes across the visual field, thereby reconciling conflicting accounts of what visual cortex predicts.

neuroscience↗

bioRxiv2026

Effects of expectation, attention, and NMDA receptor blockade on feedforward and feedback processing

Perception is increasingly viewed as an inferential process wherein sensory inputs are integrated with prior expectations. We employed time-resolved decoding on electroencephalography (EEG) data (n = 30 male participants) to investigate how expectations modulate sensory processing across varying levels of stimulus complexity, and tested the effect of attention and NMDA receptor blockade. We designed a visual stimulus containing features of different complexity whose processing relies on distinct neural mechanisms: local contrast, collinearity, and the Kanizsa illusion, involving primarily feedforward, lateral, and feedback processes, respectively. EEG decoding revealed that expectations modulated lateral and feedback processing (better decoding for unexpected stimuli) but not feedforward processing. These expectation effects were confined to attended (task-relevant) features and were not observed for task-irrelevant features. The NMDA receptor antagonist memantine selectively enhanced decoding of the Kanizsa illusion, implicating NMDA-mediated feedback mechanisms in perceptual inference, but it did not modulate the effects of expectation or attention. These findings highlight the differential impact of expectations across different stages of sensory processing and reveal a distinct role of NMDA receptor-mediated feedback mechanisms. Significance statementPerception integrates sensory inputs with prior expectations. Using EEG decoding, we examined how expectations shape sensory processing at different levels of complexity and tested the effects of attention and NMDA receptor blockade. Our visual stimuli were designed to capture EEG markers of feedforward, lateral, and feedback mechanisms. Expectations influenced lateral and feedback processing (better decoding for unexpected stimuli) but not feedforward processing, and these effects were selective to task-relevant stimulus features. Memantine, an NMDA receptor antagonist, selectively improved decoding of the Kanizsa illusion, implicating NMDA-mediated feedback in perceptual inference, but it did not shape expectation or attention effects.

neuroscience↗

bioRxiv2025

Representational magnitude as a geometric signature ofimage and word memorability

What makes some stimuli more memorable than others? Recent work has shown that image memorability is predicted by the magnitude of population responses in both monkey inferotemporal cortex and convolutional neural networks, suggesting that stimuli that more strongly activate distributed feature representations are more likely to be remembered. However, it remains unclear whether this effect is specific to the visual domain or reflects a more general property of distributed representations. Here, we show that representational magnitude predicts memorability beyond the visual domain: the effect not only replicates for images in an independent dataset, but extends to lexical memory. Across three large-scale word datasets, word embedding magnitude reliably predicts recognition memorability, independent of word frequency, valence, or length. However, the effect did not extend to a recently published voice memorability dataset, possibly reflecting distinct features driving auditory memory. Together, the cross-domain generalization suggests that the representational magnitude effect is a general property of distributed representations. Memorability, on this view, is inherent to encoding: stimuli that activate more features, and activate them more strongly - whether in brains or artificial neural networks - leave a larger representational footprint, and therefore a more lasting memory trace.

neuroscience↗

bioRxiv2025

Higher-level spatial prediction in natural vision across mouse visual cortex

Theories of predictive processing propose that sensory systems constantly predict incoming signals, based on spatial and temporal context. However, evidence for prediction in sensory cortex largely comes from artificial experiments using simple, highly predictable stimuli, that arguably encourage prediction. Here, we test for sensory prediction during natural scene perception. Specifically, we use deep generative modelling to quantify the spatial predictability of receptive field (RF) patches in natural images, and compared those predictability estimates to brain responses in the mouse visual cortex - while rigorously accounting for established tuning to a rich set of low-level image features and their local statistical context -- in a large scale survey of high-density recordings from the Allen Institute Brain Observatory. This revealed four insights. First, cortical responses across the mouse visual system are shaped by sensory predictability, with more predictable image patches evoking weaker responses. Secondly, visual cortical neurons are primarily sensitive to the predictability of higher-level image features, even in neurons in the primary visual areas that are preferentially tuned to low-level visual features. Third, unpredictability sensitivity is stronger in the superficial layers of primary visual cortex, in line with predictive coding models. Finally, these spatial prediction effects are independent of recent experience, suggesting that they rely on long-term priors about the structure of the visual world. Together, these results suggest visual cortex might predominantly predict sensory information at higher levels of abstraction - a pattern bearing striking similarities to recent, successful techniques from artificial intelligence for predictive self-supervised learning.

neuroscience↗

bioRxiv2025

Stimulus dependencies---rather than next-word prediction---can explain pre-onset brain encoding during natural listening

The human brain is thought to constantly predict future words during language processing. Recently, a new approach emerged that aims to capture neural prediction directly by using vector representations of words (embeddings) to predict brain activity prior to word onset. Two findings have been proposed as hallmarks of neural next-word prediction: (i) significant encoding prior to word onset and (ii) its modulation by word predictability. However, natural language is rife with temporal correlations, where upcoming words share statistical information with preceding ones. This raises a critical question: do these these hallmarks emerge from the brain actively predicting future content, or might they be equally well explained by the regression model exploiting these inherent stimulus dependencies? To distinguish between these alternatives, we applied the same encoding analysis to passive control systems, i.e., representational systems that encode the stimulus but cannot predict upcoming words. We show that both hallmarks emerge in two such control systems, namely in word embeddings themselves and in speech acoustics. We further show that proposed methods to correct for these dependencies are insufficient, as the effects persist even after such corrections. Together, these results suggests that pre-onset prediction of brain activity might reflect dependencies in natural language rather than predictive computations. This questions the extent to which this new encoding-based method can be used to study prediction in the brain.

neuroscience↗

bioRxiv2023

Modulation of motion signals across the visual cortical hierarchy during bistable perception

Top-down influences play a central role in perception. In multi-stable sensory phenomena subjective interpretation fluctuates spontaneously over time and the perceptual switches reflect internally driven representations engaging perceptual decision-making. Using functional magnetic resonance imaging (fMRI), we investigated how perceptual states are represented across early visual areas during perception of a bistable moving plaid. While subjects maintained central fixation, perception of the stimulus--two superimposed, obliquely oriented moving gratings--spontaneously alternated between component motion (2 gratings sliding obliquely over each other) and pattern motion (a single coherent pattern moving laterally). Direction-selective clusters were reliably modulated by perceptual state across both early (V1) and higher-order (V2 and the human medial temporal complex (hMT+)) visual regions. Pattern modulation was significantly stronger in primary visual cortex (V1) than in higher-order areas, suggesting that coherent percept formation engages mechanisms beyond simple bottom-up feature integration. Enhanced V1 modulation points to a prominent role for top-down influences from higher-order regions, consistent with predictive coding frameworks in which higher-level areas convey predictions to lower levels to disambiguate sensory input and drive perceptual inference under ambiguous conditions. Eye tracking confirmed that reflexive eye movement direction did not predict perceptual state, excluding an oculomotor confound. Interpretation of the results in light of current knowledge of the physiology of motion integration in early visual areas suggests circuits at which feedback would be predicted to act.

neuroscience↗

bioRxiv2022

Cortical activity during naturalistic music listening reflects short-range predictions based on long-term experience

Expectations shape our experience of music. However, the internal model upon which listeners form melodic expectations is still debated. Do expectations stem from Gestalt-like principles or statistical learning? If the latter, does long-term experience play an important role, or are short-term regularities sufficient? And finally, what length of context informs contextual expectations? To answer these questions, we presented human listeners with diverse naturalistic compositions from Western classical music, while recording neural activity using MEG. We quantified note-level melodic surprise and uncertainty using various computational models of music, including a state-of-the-art transformer neural network. A time-resolved regression analysis revealed that neural activity over fronto-temporal areas tracked melodic surprise particularly around 200 ms and 300-500 ms after note onset. This neural surprise response was dissociated from sensory-acoustic and adaptation effects. Neural surprise was best predicted by computational models that incorporated long-term statistical learning - rather than by simple, Gestalt-like principles. Yet, intriguingly, the surprise reflected primarily short-range musical contexts of less than ten notes. We present a full replication of our novel MEG results in an openly available EEG dataset. Together, these results elucidate the internal model that shapes melodic predictions during naturalistic music listening.

neuroscience↗

bioRxiv2021

Prediction and preview strongly affect reading times but not skipping during natural reading

In a typical text, readers look much longer at some words than at others and fixate some words multiple times, while skipping others altogether. Historically, researchers explained this variation via low-level visual or oculomotor factors, but today it is primarily explained in terms of cognitive factors, such as how well word identity can be predicted from context or discerned from parafoveal preview. While the existence of these effects has been well established in experiments, the relative importance of prediction, preview and low-level factors for eye movement variation in natural reading is unclear. Here, we address this question in three large datasets (n=104, 1.5 million words), using a deep neural network and Bayesian ideal observer to model linguistic prediction and parafoveal preview from moment to moment in natural reading. Strikingly, neither prediction nor preview was important for explaining word skipping - the vast majority of skipping was explained by a simple oculomotor model. For reading times, by contrast, we found strong but independent contributions of both prediction and preview, with effect sizes matching those from controlled experiments. Together, these results challenge dominant models of eye movements in reading by showing that linguistic prediction and parafoveal preview are not important determinants of word skipping.

neuroscience↗

bioRxiv2020

A hierarchy of linguistic predictions during natural language comprehension

Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analysing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous, probabilistic predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable signatures of syntactic, phonemic and semantic predictions. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction.

neuroscience↗

bioRxiv2018

Subjective confidence reveals the hierarchical nature of learning under uncertainty

Hierarchical processing is pervasive in the brain, but its computational significance for learning in real-world conditions, with uncertainty and changes, is disputed. We show that previously proposed qualitative signatures which relied on reports of learned quantities or choices in simple experiments are insufficient to categorically distinguish hierarchical from non-hierarchical models of learning under uncertainty. Instead, we present a novel test which leverages a more complex task, whose hierarchical structure allows generalization between different statistics tracked in parallel. We use reports of confidence to quantitatively and qualitatively arbitrate between the two accounts of learning. Our results indicate that human subjects can track multiple, interdependent levels of uncertainty, and provide clear evidence for hierarchical processing, thereby challenging some influential neurocomputational accounts of learning.

neuroscience↗