Biology subjects

Cichy, R. M.

Publications and source records attributed to Cichy, R. M..

At least 19 recordsLinked to original sources

bioRxiv2026

Clutter does not affect the spatial tuning of * category representations in human object-selective cortex *

Natural vision is frequently impaired when the visual field is crowded with multiple competing objects. One common interpretation of this clutter interference is that large receptive fields in high-level visual cortex make it hard to separate objects from their surround. This view predicts less interference for spatially localized representations; therefore, classifiers trained under clutter might rely more on such localized representations. Here we tested this by measuring participants brain signals with fMRI while they viewed objects presented either alone or within cluttered arrays. Consistent with prior work, category decoding accuracy was significantly reduced in clutter. However, classifiers trained directly on cluttered displays were not spatially more localized. Instead, they generalized robustly across spatial positions, comparable to classifiers trained on isolated objects. This finding challenges the idea that clutter preferentially disrupts position-tolerant representations.

neuroscience↗

bioRxiv2026

Language-aligned models and structured scene descriptions reveal sensitivity to compositional scene structure in the high-level visual cortex

Natural scenes are defined not only by their contained objects, but also by the structured relations among those objects. Using 7T fMRI data from the Natural Scenes Dataset, we tested whether the high-level visual cortex is sensitive to this compositional scene structure. We related narrative scene descriptions to cortical responses in an encoding framework, contrasting intact narrative descriptions with lexical control descriptions that preserved word content while disrupting compositional structure via word randomization. Intact narrative descriptions better predicted cortical responses in high-level visual cortex, indicating sensitivity to structured scene information beyond lexical content alone. This advantage was particularly pronounced for scenes with richer compositional structure. Further, a language-aligned vision model outperformed a self-supervised vision-only model, and this advantage showed a similar high-level cortical distribution. These findings suggest that the high-level visual cortex represents structured scene information beyond simple entity co-occurrence and language supervision may help induce similar representations in artificial vision models.

neuroscience↗

bioRxiv2025

Auditory representations of words during silent visual reading

Silent visual reading is accompanied by the phenomenological experience of an inner voice. However, the temporal dynamics and functional role of the underlying neural representations remain unclear. Here, we recorded electroencephalography (EEG) data while humans read naturalistic narratives, and applied computational modelling to isolate time-resolved auditory from visual and semantic representations. Our results revealed robust auditory representations during silent reading that were not explained by visual or semantic features, emerging already before word onset. These auditory representations mimicked the sequence of sounds of the corresponding words in a fine-grained manner, revealing a candidate basis for the phenomenological experience of hearing an inner voice while reading. Finally, we show that auditory word representations exhibit a key signature of predictive processing: they are stronger for unexpected than expected words. More specifically, early auditory features contribute to predictions before word onset, whereas later features only contribute after word onset, suggesting distinct prediction stages. Together our results reveal the temporal dynamics and functional role of auditory representations in silent reading.

neuroscience↗

bioRxiv2025

The Human Brain as a Dynamic Mixture of Expert Models in Video Understanding

AO_SCPLOWBSTRACTC_SCPLOWThe human brain is the most efficient and versatile system for processing dynamic visual input. By comparing representations from deep video models to brain activity, we can gain insights into mechanistic solutions for effective video processing, important to better understand the brain and to build better models. Current works in model-brain alignment primarily focus on fMRI measurements, leaving open questions about fine-grained dynamic processing. Here, we introduce the first large-scale model benchmarking on alignment to dynamic electroencephalography (EEG) recordings of short natural videos. We analyze 100+ models across the axes of temporal integration, classification task, architecture, and pretraining, using our proposed Cross-Temporal Representational Similarity Analysis (CT-RSA) which matches the best time-unfolded model features to dynamically evolving brain responses, distilling 107 alignment scores. Our findings reveal novel insights on how continuous visual input is integrated in the brain, beyond the standard temporal processing hierarchy from low to high-level representations. After initial alignment to hierarchical static object processing, responses in posterior electrodes best align to mid-level temporally-integrative action features, showing high temporal correspondence to feature timings. In contrast, responses in frontal electrodes best align with high-level static action representations and show no temporal correspondence to the video. Additionally, temporally-integrating state-space models show superior alignment to intermediate posterior activity, in which self-supervised pretraining is also beneficial. We draw a metaphor to a dynamic mixture of expert models for the changing neural preference in tasks and temporal integration reflected in the alignment to different model types across time. We posit that a single best-aligned model would need such training and architecture as to allow combining and dynamically switching between these capacities.

neuroscience↗

bioRxiv2025

Adaptive recruitment of cortex-wide recurrence for visual object recognition

Theories of the neural mechanism underpinning rapid recognition debate whether it relies solely on a feedforward sweep through the ventral stream or instead requires recurrent processing, possibly engaging additional brain regions. Here we directly tested the "adaptive recurrence hypothesis", that attempts to unify these disparate views by proposing that additional recurrent cortical resources beyond the visual stream are recruited when feedforward processing alone is insufficient to solve object recognition. To investigate this hypothesis, we contrasted functional MRI (fMRI) and electroencephalography (EEG) responses to compare neural responses to images that are equally well recognized by humans, but that differ in whether they could be solved by a feedforward deep neural network; a computational proxy for ventral stream feedforward processing. We found that when feedforward processing in the ventral visual stream is insufficient, additional parieto-frontal networks are rapidly and transiently recruited, representationally reconfiguring the ventral visual stream. Our results reveal that object recognition flexibly adapts through fast, cortex-wide recurrence, providing a unifying framework for competing theories of visual recognition.

neuroscience↗

bioRxiv2025

How do visual and conceptual factors predict the composition of typical scene drawings?

Imagine you are asked to draw a typical bedroom, what would you put on paper? Your choice of objects is likely to depend on visual occurrence statistics (i.e., the objects present in previously encountered bedrooms) and semantic relations between objects and scenes (i.e., the semantic relationship between the bedroom and its constituent objects). To investigate how these two factors contribute to the composition of typical scene drawings, we analyzed 1,192 drawings of six indoor scene categories, obtained from 303 participants. For each object featured in the drawings, we estimated its visual occurrence frequency from the ADE20K dataset of annotated scene images, and its semantic relatedness to the scene concept from a word2vec language processing model. Across all scenes of a given category, generalized linear models revealed that visual and conceptual factors both predicted the likelihood of an object featuring in the scene drawings, with a combined model outperforming both single-factor models. We further computed the visual and semantic specificity of objects for a given scene, that is, how diagnostic an object is for the scene. Object specificity offered only weak predictive power when predicting the selection of objects, yet even infrequently drawn objects remained diagnostic of their scenes. Taken together, we show that visual and conceptual factors jointly shape the composition of typical scene drawings. By releasing a large dataset of typical scene drawings alongside this work, we further provide a starting point for future studies exploring other critical properties of human drawings.

neuroscience↗

bioRxiv2025

Does Human-Like Contextual Object Recognition Emerge from Language Supervision and Language-Guided Inference?

Human vision is an active, context-sensitive process that interprets objects in relation to their surroundings. While behavioral research has long shown that scene context facilitates object recognition, the underlying computational mechanisms--and the extent to which artificial vision models replicate this ability--remain unclear. Here, we addressed this gap by combining human behavioral experiments with computational modeling to investigate how structured scene context influences object recognition. Using a novel 3D simulation framework, we embedded target objects into indoor scenes, and manipulated contextual coherence between objects and scenes by using either intact scenes or their phase-scrambled versions. Humans showed a robust object recognition advantage in coherent scenes, particularly under challenging conditions such as occlusion, crowding, or non-canonical viewpoints. Conventional vision models--including convolutional neural networks (CNNs) and vision transformers (ViTs)--failed to replicate this effect. In contrast, vision-language models (VLMs), particularly those using ViT architectures and trained with language supervision (e.g., CLIP), approached human-like accuracy. This shows that semantically rich and category-structured representations are required for modelling context sensitivity. Notably, context sensitive behavior was closest to humans in VLMs when using language-guided inference at test time. This suggests that how a model accesses its representations during inference is relevant for enabling context-sensitive behavior. Together, this work offers steps towards a computational account of contextual facilitation of objects by scenes, and highlights zero-shot inference as an interesting alignment metric when benchmarking artificial and biological vision.

neuroscience↗

bioRxiv2025

Layer-specific spatiotemporal dynamics of feedforward and feedback in human visual object perception

Visual object perception is mediated by information flow between regions of the ventral visual stream along feedforward and feedback anatomical connections. However, feedforward and feedback signals during naturalistic vision are rapid and overlapping, complicating their identification and precise functional specification. Here we recorded human layer-specific fMRI responses to naturalistic object images in early visual cortex (EVC) and lateral occipital complex (LOC) to isolate feedforward and feedback information signals spatially by their cortical layer-specific termination pattern. We combined these layer-specific fMRI responses with electroencephalography (EEG) responses for the same images to segregate feedforward and feedback signals in both time and space. Feedforward signals emerge early in the middle layers of EVC and LOC, followed by feedback signals in the superficial layer of LOC. Comparing the identified dynamics in LOC to a visual deep neural network (DNN), revealed that early feedforward signals in LOC encode medium-to-high complexity features, whereas later feedback signals increase the representational format to high complexity features. Together this specifies the spatiotemporal dynamics and functional role of feedforward and feedback information flow mediating visual object perception.

neuroscience↗

bioRxiv2025

Absence of Systematic Effects of Internalizing Psychopathology on Learning Under Uncertainty

Difficulties in adapting learning to meet the challenges of uncertain and changing environments are widely thought to play a central role in internalizing psychopathology, including anxiety and depression. This view stems from findings linking trait anxiety and transdiagnostic internalizing symptoms to learning impairments in laboratory tasks often used as proxies for real-world behavioral flexibility. These tasks typically require learners to adjust learning rates dynamically in response to uncertainty, for instance, increasing learning from prediction errors in volatile environments. However, prior studies have produced inconsistent and sometimes contradictory findings regarding the nature and extent of learning impairments in populations with internalizing disorders. To address this, we conducted eight experiments (N = 820) using predictive inference and reversal learning tasks, and applied a bi-factor analysis to capture internalizing symptom variance shared across and differentiated between anxiety and depression. While we observed robust evidence for adaptive learning-rate modulation across participants, we found no convincing evidence of a systematic relationship between internalizing symptoms and either learning rates or task performance. These findings challenge prominent claims that learning difficulties are a hallmark feature of internalizing psychopathology and suggest that the relationship between these traits and adaptive behavior under uncertainty may be more subtle than previously thought.

neuroscience↗

bioRxiv2025

Investigating the temporal dynamics and modelling of mid-level feature representations in humans

Scene perception is a key function of biological visual systems. According to the hierarchical processing view, scene perception in the human brain begins with low-level features, progresses to mid-level features, and ends with high-level features. While low- and high-level feature processing is well-studied, research on mid-level features remains limited. Here, we addressed this gap by investigating when mid-level features are processed in humans using a novel stimulus set of naturalistic scenes as images and videos, accompanied with ground-truth annotations for five mid-level features (reflectance, lighting, world normals, scene depth and skeleton position), and two framing features: one low-level (edges) and one high-level feature (action). To reveal when low-, mid- and high-level features are represented in the brain, we collected electroencephalography (EEG) data from human participants during stimulus presentation and trained encoding models to predict EEG data from ground-truth annotations. We revealed that mid-level features were best represented between [~]100 and [~]250 ms post-stimulus, between low- and high-level features. Moreover, we assessed scene- and action-trained convolutional neural networks (CNNs) as models of mid-level feature processing in humans. We found a comparable processing order for mid-but not low- or high-level features with humans. Overall, our results characterize mid-level feature processing in humans in the temporal domain and reveal CNNs as suitable models of the processing hierarchy of mid-level vision in humans.

neuroscience↗

bioRxiv2025

Hippocampal-cortical connectivity relates to inter-individual differences and training gains in distinguishing similar memories

Mnemonic discrimination (MD) is the ability to distinguish current experiences from similar memories. Research on the brain correlates of MD has focused on how regional neural responses are linked to MD. Here we go beyond this approach to investigate inter-regional functional connectivity patterns related to MD, its inter-individual variability and training-related improvement. Based on prior work we focused on medial temporal lobe (MTL), prefrontal cortex (PFC) and visual regions. We used fMRI to determine how functional connectivity patterns between these regions are related to MD before and after 2-weeks of web-based cognitive training. We identified a functional connectivity signature involving MTL-PFC-visual areas during successful MD. We found that hippocampal-PFC connectivity was negatively associated with interindividual variability in MD performance across two different tasks. Hippocampal-PFC connectivity decrease was also linked to interindividual variability in post-training MD improvement. Additionally, training led to increased connectivity from the lateral occipital cortex to the occipital pole area. Our results point to a hippocampal-PFC connectivity pattern which is a reliable, task-invariant, marker of MD performance. This pattern is further related to MD training gains providing causal evidence for its relevance in distinguishing similar memories. Overall, we show that hippocampal-PFC connectivity constitutes a neural resource for MD that enables training-related improvements and could be targeted in future research to enhance cognition.

neuroscience↗

bioRxiv2024

Touch to text: Spatiotemporal evolution of braille letter representations in blind readers

Visual deprivation does not silence the visual cortex, which is responsive to auditory, tactile, and other nonvisual tasks in blind persons. However, the underlying functional dynamics of the neural networks mediating such crossmodal responses remain unclear. Here, using braille reading as a model framework to investigate these networks, we presented sighted (N=13) and blind (N=12) readers with individual visual print and tactile braille alphabetic letters, respectively, during MEG recording. Using time-resolved multivariate pattern analysis and representational similarity analysis, we traced the alphabetic letter processing cascade in both groups of participants. We found that letter representations unfolded more slowly in blind than in sighted brains, with decoding peak latencies [~]200 ms later in braille readers. Focusing on the blind group, we found that the format of neural letter representations transformed within the first 500 ms after stimulus onset from a low-level structure consistent with peripheral nerve afferent coding to high-level format reflecting pairwise letter embeddings in a text corpus. The spatiotemporal dynamics of the transformation suggest that the processing cascade proceeds from a starting point in somatosensory cortex to early visual cortex and then to inferotemporal cortex. Together our results give insight into the neural mechanisms underlying braille reading in blind persons and the dynamics of functional reorganization in sensory deprivation.

neuroscience↗

bioRxiv2024

Enhanced and idiosyncratic neural representations of personally typical scenes

Previous research shows that the typicality of visual scenes (i.e., if they are good examples of a category) determines how easily they can be perceived and represented in the brain. However, the unique visual diets individuals are exposed to across their lifetimes should sculpt very personal notions of typicality. Here, we thus investigated whether scenes that are more typical to individual observers are more accurately perceived and represented in the brain. We used drawings to enable participants to describe typical scenes (e.g., a kitchen) and converted these drawings into 3D renders. These renders were used as stimuli in a scene categorization task, during which we recorded EEG. In line with previous findings, categorization was most accurate for renders resembling the typical scene drawings of individual participants. Our EEG analyses reveal two critical insights on how these individual differences emerge on the neural level: First, personally typical scenes yielded enhanced neural representations from around 200 ms after onset. Second, personally typical scenes were represented in idiosyncratic ways, with reduced dependence on high-level visual features. We interpret these findings in a predictive processing framework, where individual differences in internal models of scene categories formed through experience shape visual analysis in idiosyncratic ways.

neuroscience↗

bioRxiv2024

Healthy aging delays and dedifferentiates high-level visual representations

1.Healthy aging impacts visual information processing with consequences for subsequent high-level cognition and everyday behavior, but the underlying neural changes in visual representations remain unknown. Here, we investigate the nature of representations underlying object recognition in older compared to younger adults by tracking them in time using EEG, across space using fMRI, and by probing their behavioral relevance using similarity judgements. Applying a multivariate analysis framework to combine experimental assessments, four key findings about how brain aging impacts object recognition emerge. First, aging selectively delays the formation of object representations, profoundly changing the chronometry of visual processing. Second, the delay in the formation of object representations emerges in high-level rather than low- and mid-level ventral visual cortex, supporting the theory that brain areas developing last deteriorate first. Third, aging reduces content selectivity in high-level ventral visual cortex, indicating age-related neural dedifferentiation as the mechanism of representational change. Finally, we demonstrate that the identified representations of the aging brain are behaviorally relevant, ascertaining ecological relevance. Together, our results reveal the impact of healthy aging on the visual brain.

neuroscience↗

bioRxiv2024

The representational nature of spatio-temporal recurrent processing in visual object recognition

The human brain orchestrates object vision through an interplay of feedforward processing in concert with recurrent processing. However, where, when and how recurrent processing contributes to visual processing is incompletely understood due to the difficulties in teasing apart feedforward and recurrent processing. We combined a backward masking paradigm with multivariate analysis on EEG and fMRI data to isolate and characterize the nature of recurrent processing. We find that recurrent processing substantially shapes visual representations across the ventral visual stream, starting early on at around 100ms in early visual cortex (EVC) and in two later phases of around 175 and 300ms in lateral occipital cortex (LOC), adding persistent rather than transient neural dynamics to visual processing. Using deep neural network models for comparison with the brain, we show that recurrence changes the feature format in LOC from predominantly mid-level to more high-level features. Finally, we show that recurrence is mediated by four distinct spectro-temporal neural components in EVC and LOC, which span the theta to beta frequency range. Together, our results reveal the nature and mechanisms of the effects of recurrent processing on the visual representations in the human brain.

neuroscience↗

bioRxiv2024

Adaptive Integration of Perceptual and Reward Information in an Uncertain World

Perceptual uncertainty and salience both impact decision-making, but how these factors precisely impact trial-and-error reinforcement learning is not well understood. Here, we test the hypotheses that (H1) perceptual uncertainty modulates reward-based learning and that (H2) economic decision-making is driven by the value and the salience of sensory information. For this, we combined computational modeling with a perceptual uncertainty-augmented reward-learning task in a human behavioral experiment (N = 98). In line with our hypotheses, we found that subjects regulated learning behavior in response to the uncertainty with which they could distinguish choice options based on sensory information (belief state), in addition to the errors they made in predicting outcomes. Moreover, subjects considered a combination of expected values and sensory salience for economic decision-making. Taken together, this shows that perceptual and economic decision-making are closely intertwined and share a common basis for behavior in the real world.

neuroscience↗

bioRxiv2024

The transformation of sensory to perceptual braille letter representations in the visually deprived brain

1.Experience-based plasticity of the human cortex mediates the influence of individual experience on cognition and behavior. The complete loss of a sensory modality is among the most extreme such experiences. Investigating such a selective, yet extreme change in experience allows for the characterization of experience-based plasticity at its boundaries. Here, we investigated information processing in individuals who lost vision at birth or early in life by probing the processing of braille letter information. We characterized the transformation of braille letter information from sensory representations depending on the reading hand to perceptual representations that are independent of the reading hand. Using a multivariate analysis framework in combination with fMRI, EEG and behavioral assessment, we tracked cortical braille representations in space and time, and probed their behavioral relevance. We located sensory representations in tactile processing areas and perceptual representations in sighted reading areas, with the lateral occipital complex as a connecting "hinge" region. This elucidates the plasticity of the visually deprived brain in terms of information processing. Regarding information processing in time, we found that sensory representations emerge before perceptual representations. This indicates that even extreme cases of brain plasticity adhere to a common temporal scheme in the progression from sensory to perceptual transformations. Ascertaining behavioral relevance through perceived similarity ratings, we found that perceptual representations in sighted reading areas, but not sensory representations in tactile processing areas are suitably formatted to guide behavior. Together, our results reveal a nuanced picture of both the potentials and limits of experience-dependent plasticity in the visually deprived brain.

neuroscience↗

bioRxiv2023

Task-Switch Related Reductions in Neural Distinctiveness in Children and Adults: Commonalities and Differences

Goal-directed behavior requires the ability to flexibly switch between task sets with changing environmental demands. Switching between tasks generally comes at the cost of slower and less accurate responses. Compared to adults, children show greater switch costs, presumably reflecting the protracted development of the ability to flexibly update task-set representations. To examine whether the distinctiveness of neural task-set representations is more strongly affected by a task switch in children compared to adults, we examined multi-voxel patterns of fMRI activation in 88 children (8-11 years, 49 girls) and 53 adults (20-30 years, 28 women) during a task-switching paradigm. Using multivariate pattern analysis (MVPA), we investigated whether task-set representations were less distinct on switch than on repeat trials across frontoparietal, cingulo-opercular, and temporo-occipital regions. Children and adults showed lower accuracy and longer response times on switch than on repeat trials, with higher accuracy costs in children. Decoding accuracy across regions was lower on switch than repeat trials, consistent with the notion that switching reduces the distinctiveness of task-set representations. Reliable age differences in switch-related representational distinctiveness reductions were absent, pointing to a remarkable degree of maturity of neural representations of task-relevant information in late childhood. However, we also observed that switch-related reductions in distinctiveness were more highly correlated across frontoparietal and cingulo-opercular regions in children than in adults, potentially reflecting the ongoing specialization of different control networks with respect to the representation of task features. Significance statementThe ability to flexibly switch between tasks enables goal-directed behavior, but is particularly challenging for children, potentially due to protracted development in the ability to represent multiple and overlapping task rules that link stimuli to appropriate responses. We tested this hypothesis by using neuroimaging to measure brain activity during task switching in 8-11-year-olds and adults. Activation patterns in frontal, parietal, and temporal regions could tell us with above-chance accuracy which task a person was performing when the task remained the same, but not when it had switched. Adults showed greater differentiation across regions in terms of switch-related reductions in distinctiveness than children, suggesting that the relevant functional circuity is present but has not yet fully matured by late childhood.

neuroscience↗