Biology subjects

Scholte, H. S.

Publications and source records attributed to Scholte, H. S..

At least 19 recordsLinked to original sources

bioRxiv2026

Brain alignment in deep neural networks emerges early and independently of object classification

Deep convolutional neural networks are leading models of biological vision, largely because of their strong brain alignment: their features predict neural responses better than earlier models. Yet they are believed to recognize objects differently, relying on texture where humans rely on shape and failing on perturbations humans handle effortlessly. What then does alignment reflect? Tracking three architectures densely during training, we find that alignment with human fMRI, EEG, and macaque electrophysiology is already largely present at initialization, when networks classify at chance, and reaches a plateau within one to five epochs; thereafter it changes only modestly while classification accuracy continues to climb to 75%. Network lesioning shows that a kernel's contribution to alignment is essentially uncorrelated with its contribution to classification throughout training. Brain-network alignment therefore appears to reflect the structure of the visual environment both systems encode, rather than a shared solution for classification.

neuroscience↗

bioRxiv2026

Learning to See Like a Child: Why Viewpoint Diversity is Fundamental for Human-Aligned Object Recognition

Deep convolutional neural networks match human accuracy on standard object recognition tasks but fail to recognize familiar objects from novel view-points. Humans, however, develop viewpoint-invariant recognition at an early age through diverse visual experience. This gap in visual experience may explain why models diverge from humans in object recognition. Holding dataset size constant, we show that greater viewpoint diversity substantially improves generalization to novel views. Using a synthetic 3D dataset with systematically controlled viewpoints, we reveal a core trade-off: restricted-view training yields rapid learning and near-ceiling in-distribution accuracy but collapses on held-out viewpoints, whereas viewpoint-diverse training learns more gradually yet generalizes robustly. Increasing viewpoint diversity disrupts texture regularities while preserving global shape, driving networks to prioritize shape over texture - the same strategy that underlies human object recognition. Partitioned Grad-CAM analyses further show that viewpoint-diverse models maintain object-centered attention. These findings parallel developmental accounts of multi-view learning and identify viewpoint diversity as an important factor for robust, human-aligned vision.

neuroscience↗

bioRxiv2026

Image-grounded encoding models reveal distinct temporal profiles of naturalistic object and scene processing in the human brain

In real-world vision, the human brain needs to process large amounts of information to effectively interact with its environment. It is well established that our visual system has specialized regions to process information efficiently, such as scene-, face-, and object-selective areas, which can be uncovered using functional magnetic resonance imaging (fMRI). However, functional localizers typically use artificially separated stimuli (e.g. scene backgrounds vs. object cutouts), and fMRI signals are insensitive to potential temporal processing differences. Here, we identify temporal signatures of object and scene processing in natural visual environments by building image-grounded brain-predictive encoding models of human electroencephalography (EEG) responses. In a large set of high-resolution natural images, we first separate object from scene information on a per-image basis, and then feed this information to train separate deep neural network-based encoding models to predict EEG responses to the full, intact images. We find that encoding models that receive only object information consistently exhibit a delayed temporal encoding profile compared to models that only receive scene information. Control analyses confirm the robustness of this delayed object encoding, showing that consistent selection of object or scene information is needed to achieve high encoding performance. Using these distinct encoding profiles as processing templates, we reveal which image parts are seen by the brain as an object, versus a background scene element. These findings demonstrate that object and scene processing in the human brain unfold differentially over time and indicate that image-grounded encoding models are powerful tools for isolating components of naturalistic perception. SignificanceThe neural mechanisms underlying visual perception are typically studied by experimentally separating different kinds of information, for example presenting either isolated objects or scene backgrounds. Since information naturally co-occurs in the world -- objects are typically embedded within scenes -- this artificial separation reduces ecological validity. We leverage image-computable neural encoding models to identify distinct visual processing profiles within ecologically valid, natural images. Models that receive stimuli in which only objects are visible and scene information is masked predict EEG responses at later time points than models receiving the inverse, "scene-only" stimuli. By quantifying how items such as buildings, trees or the ground are encoded in EEG signals, we reveal which elements of visual environments drive neural responses to intact, real-world images.

neuroscience↗

bioRxiv2026

PRISMA: A precision functional imaging dataset of autistic and non-autistic adults

Autism spectrum conditions are characterised by substantial heterogeneity in brain function, yet most neuroimaging studies rely on brief single-session acquisitions that prioritise group-level comparisons over individual characterisation. We present a precision functional imaging dataset with extended neuroimaging acquisitions to advance research on individual differences in autism. The dataset comprises 33 autistic adults between the ages of 18 and 30, and 34 age- and gender-matched comparison participants, each scanned across three sessions with up to 90 minutes of functional magnetic resonance imaging per participant. Functional data were acquired during eyes-open rest, news viewing, and reality television viewing to probe state-dependent differences in network engagement. The dataset includes high-resolution structural imaging, diffusion-weighted imaging with dual phase-encoding, and physiological recordings of cardiac and respiratory activity acquired simultaneously with functional scans. All data are organised following Brain Imaging Data Structure principles and accompanied by comprehensive quality control metrics and preprocessed derivatives. This dataset enables investigation of individual-specific functional brain organisation, test-retest reliability of connectivity patterns, and state-dependency of functional networks in autism, addressing a notable gap in publicly available neuroimaging resources.

neuroscience↗

bioRxiv2025

Shared texture-like representations, not global form, underlie deep neural network alignment with human visual processing

Deep neural networks (DNNs) excel at predicting neural responses across the visual hierarchy 1-5, a success widely interpreted as evidence of shared object recognition computations 6,7. Yet improving DNN object recognition accuracy does not reliably increase neural predictivity 8,9, and even untrained networks predict brain responses above chance 9-11. This disconnect suggests that object recognition may not drive DNN-brain alignment. Texture-like statistics are represented in both DNNs and mid-level visual cortex 12-17. In natural images, these statistics are carried by objects and backgrounds, shaping representations and recognition in both systems 18-21. Does DNN-brain alignment reflect shared sensitivity to object-related information or texture-like statistics? To dissociate these factors, we recorded EEG from 57 participants viewing natural scenes, texture-synthesized images preserving local statistics while disrupting global form and object-only images with backgrounds removed. If alignment reflects texture-like statistics, it should peak for texture-synthesized images. If it reflects object-related processing, alignment should be strongest for natural and object-only conditions, which preserve object information. We compared EEG responses with DNN activations via weighted representational similarity analysis 22,23. Texture-synthesized images yielded the strongest DNN-EEG alignment, peaking in early responses (<200 ms) and explaining up to [~]85% of noise-ceiling-normalized explainable variance versus [~]44% for natural and [~]55% for isolated objects. Crucially, object categories were more decodable for natural and object-only images than texture-synthesized images, yet these object-rich conditions showed weaker alignment. This dissociation reveals that DNNs capture the texture-statistical component of early visual responses while failing to explain later, object-related variance. HighlightsO_LITexture-synthesized images yield the strongest DNN-brain alignment across DNNs. C_LIO_LIObject categories decode better from object-rich than texture-synthesized EEG. C_LIO_LIDNN-brain alignment reflects shared texture statistics, not object-related processing. C_LI

neuroscience↗

bioRxiv2025

Scene segmentation processes drive EEG-DCNN alignment

Visual processing in biological and artificial neural networks has been extensively studied through the lens of object recognition. While deep convolutional neural networks (DCNNs) have demonstrated hierarchical feature extraction similar to biological systems (DiCarlo and Cox, 2007; Yamins and DiCarlo, 2016), recent findings reveal a growing discrepancy: DCNNs with higher object categorization accuracy paradoxically show worse performance at predicting neural responses (Xu and Vaziri-Pashkam, 2021; Linsley et al., 2023). Using a large-scale human electroencephalography (EEG) dataset (n=10, 82,160 trials), we investigate whether this discrepancy arises because human neural EEG signals predominantly reflect scene segmentation processes rather than high-level, category-specific object representations. We trained DCNNs to perform object recognition using visual diets ([~]1 million training images across 292 object categories) with systematically varying scene segmentation demands: objects-only (pre-segmented), background-silhouette (explicit boundaries), original/background-only images (requiring full segmentation). Despite substantial differences in categorization accuracy (27-53%), all trained models showed remarkably uniform encoding performance, with peak correlations with neural data at [~]0.1s post-stimulus. Layer-wise analysis revealed a significant negative correlation between categorization accuracy and encoding performance, with earlier network layers better predicting EEG responses than deeper layers specialized for object categorization. This dissociation suggests that EEG signals primarily reflect fundamental scene parsing mechanisms rather than object-specific representations, explaining the growing discrepancy between DCNNs increasing categorization performance but deteriorating neural prediction performance. Significance StatementThis research provides a novel perspective on human electroencephalography (EEG) signals during visual processing through systematic manipulation of scene segmentation demands in deep neural networks. Using a large-scale dataset of 82,160 EEG trials and 20 trained DCNNs, we demonstrate that EEG responses primarily reflect early visual processing involved in breaking down and organizing visual scenes (scene segmentation/parsing) rather than high-level object recognition. This finding helps explain previously observed discrepancies between DCNNs categorization performance and neural prediction accuracy, suggesting that improving models ability to segment scenes, rather than simply recognizing isolated objects, may better align artificial and biological visual processing.

neuroscience↗

bioRxiv2024

Spatial sampling of deep neural network features improves encoding models of foveal and peripheral visual processing in humans

Deep Neural Networks (DNNs) are increasingly being used to build encoding models to predict neural data recorded during visual stimulation. Their ability to process natural images makes them a prime candidate for studying the neural profile underlying real-world visual perception. However, there are still prominent discrepancies in how DNNs and humans process visual information. One discrepancy lies in the spatial sampling of visual input: while DNNs uniformly sample from their input at every spatial location, the human visual system samples differentially from central and peripheral regions. Here, we implement multiple spatial sampling strategies on feature maps of DNNs into encoding models that predict human EEG responses to a novel stimulus set consisting of large, high-quality natural scene images. By applying differential spatial sampling of DNN feature maps, we reveal distinct temporal profiles for encoding of peripheral vs. central information in EEG signals. Moreover, we show that a differential spatial sampling derived from the density of retinal ganglion cells yields the best performing encoding model when using DNN feature maps. We experimentally confirm this pattern by separately stimulating peripheral and central visual field regions, and demonstrate that the distinct temporal profiles for central and peripheral information are only revealed when using large-field stimuli. Together, these results show that aligning the spatial sampling of humans and DNN encoding models can improve predictions of neural data. The distinct temporal profiles for encoding of peripheral vs. central information support a global-to-local processing hierarchy of real-world vision.

neuroscience↗

bioRxiv2024

Shape-Biased Learning by Thinking Inside the Box

Convolutional Neural Networks (CNNs) surpass human-level performance on visual object recognition and detection, but their behavior still differs from human behavior in important ways. One prominent example is that CNNs trained on ImageNet exhibit an image texture bias, while humans exhibit a strong bias toward object shape. Although CNN shape bias can be increased in various ways, e.g., using data augmentation or additional training techniques, it remains unclear what causes the strong discrepancy between human and CNN object recognition strategies. Developmental research suggests that one factor driving human shape bias is that during early childhood, toddlers tend to fill their field-of-view with close-up objects. Here, we operationalize this close-up as a zoom-in on objects during CNN training which we show increases shape bias without any additional training or data augmentation. We provide further evidence for the advantage of closeup object vision by systematically manipulating the background-object ratio during CNN training, and demonstrate a strong (inverse) correlation with shape bias. Moreover, zooming-in on objects, thereby more closely emulating child vision, not only increases shape bias but also concurrently aligns classification accuracy and shape bias between humans and CNNs. Finally, we achieve a near human-like shape bias when using a developmentally-inspired background-object ratio for training and shape bias assessment. In sum, from a simple adjustment to common image datasets - zooming-in on objects - human-like shape bias can emerge. These results suggest that taking inspiration from human learning strategies is a promising avenue for building human-aligned, efficient, and more robust vision CNNs.

neuroscience↗

bioRxiv2023

Human visual cortex and deep convolutional neural network care deeply about object background

Deep convolutional neural networks (DCNNs) are able to predict brain activity during object categorization tasks, but factors contributing to this predictive power are not fully understood. Our study aimed to investigate the factors contributing to the predictive power of DCNNs in object categorization tasks. We compared the activity of four DCNN architectures with electroencephalography (EEG) recordings obtained from 62 human subjects during an object categorization task. Previous physiological studies on object categorization have highlighted the importance of figure-ground segregation - the ability to distinguish objects from their backgrounds. Therefore, we set out to investigate if figure-ground segregation could explain DCNNs predictive power. Using a stimuli set consisting of identical target objects embedded in different backgrounds, we examined the influence of object background versus object category on both EEG and DCNN activity. Crucially, the recombination of naturalistic objects and experimentally-controlled backgrounds creates a sufficiently challenging and naturalistic task, while allowing us to retain experimental control. Our results showed that early EEG activity (<100ms) and early DCNN layers represent object background rather than object category. We also found that the predictive power of DCNNs on EEG activity is related to processing of object backgrounds, rather than categories. We provided evidence from both trained and untrained (i.e. random weights) DCNNs, showing figure-ground segregation to be a crucial step prior to the learning of object features. These findings suggest that both human visual cortex and DCNNs rely on the segregation of object backgrounds and target objects in order to perform object categorization. Altogether, our study provides new insights into the mechanisms underlying object categorization as we demonstrated that both human visual cortex and DCNNs care deeply about object background. Author summaryOur study aimed to investigate the factors contributing to the predictive power of deep convolutional neural networks (DCNNs) on EEG activity in object recognition tasks. We compared the activity of four DCNN architectures with human neural recordings during an object categorization task. We used a stimuli set consisting of identical target objects embedded in different phase-scrambled backgrounds. The distinction between object backgrounds and object categories allows us to investigate the influence of either factor for human subjects and DCNNs. Surprisingly, we found that both human visual processing and early DCNNs layers dedicate a large proportion of activity to processing object backgrounds instead of object category. Furthermore, this shared ability to make object backgrounds (and not just object category) invariant is largely the reason why DCNNs are predictive of brain dynamics in our experiment. We posit this shared ability to be an important solution for object categorization. Finally, we conclude that DCNNs, like humans, care deeply about object backgrounds.

neuroscience↗

bioRxiv2022

Mechanisms of human dynamic object recognition revealed by sequential deep neural networks

Humans can rapidly recognize objects in a dynamically changing world. This ability is showcased by the fact that observers succeed at recognizing objects in rapidly changing image sequences, at up to 13 ms/image. To date, the mechanisms that govern dynamic object recognition remain poorly understood. Here, we developed deep learning models for dynamic recognition and compared different computational mechanisms, contrasting feedforward and recurrent, single-image and sequential processing as well as different forms of adaptation. We found that only models that integrate images sequentially via lateral recurrence mirrored human performance (N=36) and were predictive of trial-by-trial responses across image durations (13-80 ms/image) while also displaying a temporal correspondence. Augmenting this model with adaptation markedly improved dynamic recognition and accelerated its representational dynamics, thereby predicting human trial-by-trial responses using fewer processing resources. These findings provide new insights into the mechanisms rendering object recognition so fast and effective in a dynamic visual world.

neuroscience↗

bioRxiv2022

A critical test of deep convolutional neural networks ability to capture recurrent processing in the brain using visual masking

Recurrent processing is a crucial feature in human visual processing supporting perceptual grouping, figure-ground segmentation, and recognition under challenging conditions. There is a clear need to incorporate recurrent processing in deep convolutional neural networks (DCNNs) but the computations underlying recurrent processing remain unclear. In this paper, we tested a form of recurrence in deep residual networks (ResNets) to capture recurrent processing signals in the human brain. Though ResNets are feedforward networks, they approximate an excitatory additive form of recurrence. Essentially, this form of recurrence consists of repeating excitatory activations in response to a static stimulus. Here, we used ResNets of varying depths (reflecting varying levels of recurrent processing) to explain electroencephalography (EEG) activity within a visual masking paradigm. Sixty-two humans and fifty artificial agents (10 ResNet models of depths - 4, 6, 10, 18 and 34) completed an object categorization task. We show that deeper networks (ResNet-10, 18 and 34) explained more variance in brain activity compared to shallower networks (ResNet-4 and 6). Furthermore, all ResNets captured differences in brain activity between unmasked and masked trials, with differences starting at [~]98ms (from stimulus onset). These early differences indicated that EEG activity reflected pure feedforward signals only briefly (up to [~]98ms). After [~]98ms, deeper networks showed a significant increase in explained variance which peaks at [~]200ms, but only within unmasked trials, not masked trials. In summary, we provided clear evidence that excitatory additive recurrent processing in ResNets captures some of the recurrent processing in humans. Significance statementThe challenge of modeling recurrent processes is not trivial and the operationalization of recurrent processing is highly contested. In this paper, we tested the ability of deep residual networks (ResNets) to explain recurrent processes in the human brain. Though ResNets are feedforward networks, they have been shown to equate operations in recurrent neural networks. In this study, we show that deeper networks explained more variance in brain activity than shallower networks. However, all networks still performed far from the noise ceiling. Thus, we conclude that recurrent processing in ResNets captures a form of recurrent processing in humans though other types of recurrent processing (inhibition, multiplicative) that are not present in current regular deep neural networks (alexnet, cornet, resnet) are necessary for building better visual models.

neuroscience↗

bioRxiv2021

Deep hierarchical sensory processing accounts for effects of arousal state on perceptual decision-making

Arousal levels strongly affect task performance. Yet, what arousal level is optimal for a task depends on its difficulty. Easy task performance peaks at higher arousal levels, whereas performance on difficult tasks displays an inverted U-shape relationship with arousal, peaking at medium arousal levels, an observation first made by Yerkes and Dodson in 1908. It is commonly proposed that the noradrenergic locus coeruleus system regulates these effects on performance through a widespread release of noradrenaline resulting in changes of cortical gain. This account, however, does not explain why performance decays with high arousal levels only in difficult, but not in simple tasks. Here, we present a mechanistic model that revisits the Yerkes-Dodson effect from a sensory perspective: a deep convolutional neural network augmented with a global gain mechanism reproduced the same interaction between arousal state and task difficulty in its performance. Investigating this model revealed that global gain states differentially modulated sensory information encoding across the processing hierarchy, which explained their differential effects on performance on simple versus difficult tasks. These findings offer a novel hierarchical sensory processing account of how, and why, arousal state affects task performance. Authors summaryOver a hundred years ago, it was first observed that the effect of arousal on performance depends on task difficulty: the Yerkes-Dodson effect. Difficult tasks are best solved at intermediate arousal levels, whereas easy tasks benefit from a high arousal state. Current theories on how arousal affects neural processing cannot explain this effect of task difficulty. Here, we implement a key effect of arousal on cortical processing, a change in neuronal gain, in a computational model of visual processing capable of object recognition. Across a series of experiments, we find that our model can reproduce the Yerkes-Dodson effect behaviorally and that this effect can be explained by where in the processing hierarchy different arousal states optimize sensory information encoding.

neuroscience↗

bioRxiv2020

Leveraging spiking deep neural networks tounderstand neural mechanisms underlyingselective attention

Spatial attention enhances sensory processing of goal-relevant information and improves perceptual sensitivity. Yet, the specific neural mechanisms underlying the effects of spatial attention on performance are still contested. Here, we examine different attention mechanisms in spiking deep convolutional neural networks. We directly contrast effects of precision (internal noise suppression) and two different gain modulation mechanisms on performance on a visual search task with complex real-world images. Unlike standard artificial neurons, biological neurons have saturating activation functions, permitting implementation of attentional gain as gain on a neurons input or on its outgoing connection. We show that modulating the connection is most effective in selectively enhancing information processing by redistributing spiking activity, and by introducing additional task-relevant information, as shown by representational similarity analyses. Precision only produced minor attentional effects in performance. Our results, which mirror empirical findings, show that it is possible to adjudicate between attention mechanisms using more biologically realistic models and natural stimuli.

neuroscience↗

bioRxiv2020

On the necessity of recurrent processing during object recognition: it depends on the need for scene segmentation

While feed-forward activity may suffice for recognizing objects in isolation, additional visual operations that aid object recognition might be needed for real-world scenes. One such additional operation is figure-ground segmentation; extracting the relevant features and locations of the target object while ignoring irrelevant features. In this study of 60 participants, we show objects on backgrounds of increasing complexity to investigate whether recurrent computations are increasingly important for segmenting objects from more complex backgrounds. Three lines of evidence show that recurrent processing is critical for recognition of objects embedded in complex scenes. First, behavioral results indicated a greater reduction in performance after masking objects presented on more complex backgrounds; with the degree of impairment increasing with increasing background complexity. Second, electroencephalography (EEG) measurements showed clear differences in the evoked response potentials (ERPs) between conditions around 200ms - a time point beyond feed-forward activity and object decoding based on the EEG signal indicated later decoding onsets for objects embedded in more complex backgrounds. Third, Deep Convolutional Neural Network performance confirmed this interpretation; feed-forward and less deep networks showed a higher degree of impairment in recognition for objects in complex backgrounds compared to recurrent and deeper networks. Together, these results support the notion that recurrent computations drive figure-ground segmentation of objects in complex scenes.

neuroscience↗

bioRxiv2020

The Amsterdam Open MRI Collection, a set ofmultimodal MRI datasets for individual differenceanalyses

We present the Amsterdam Open MRI Collection (AOMIC): three datasets with multimodal (3T) MRI data including structural (T1-weighted), diffusion-weighted, and (resting-state and task-based) functional BOLD MRI data, as well as detailed demographics and psychometric variables from a large set of healthy participants (N = 928, N = 226, and N = 216). Notably, task-based fMRI was collected during various robust paradigms (targeting naturalistic vision, emotion perception, working memory, face perception, cognitive conflict and control, and response inhibition) for which extensively annotated event-files are available. For each dataset and data modality, we provide the data in both raw and preprocessed form (both compliant with the Brain Imaging Data Structure), which were subjected to extensive (automated and manual) quality control. All data is publicly available from the Openneuro data sharing platform.

neuroscience↗

bioRxiv2019

Depth in convolutional neural networks solves scene segmentation

Feedforward deep convolutional neural networks (DCNNs) are, under specific conditions, matching and even surpassing human performance in object recognition in natural scenes. This performance suggests that the analysis of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to solve specific visual subtasks. Research in humans however suggests that while feedforward activity may suffice for sparse scenes with isolated objects, additional visual operations ( routines) that aid the recognition process (e.g. segmentation or grouping) are needed for more complex scenes. Linking human visual processing to performance of DCNNs with increasing depth, we here explored if, how, and when object information is differentiated from the backgrounds they appear on. To this end, we controlled the information in both objects and backgrounds, as well as the relationship between them by adding noise, manipulating background congruence and systematically occluding parts of the image. Results indicate that with an increase in network depth, there is an increase in the distinction between object- and background information. For more shallow networks, results indicated a benefit of training on segmented objects. Overall, these results indicate that, de facto, scene segmentation can be performed by a network of sufficient depth. We conclude that the human brain could perform scene segmentation in the context of object identification without an explicit mechanism, by selecting or "binding" features that belong to the object and ignoring other features, in a manner similar to a very deep convolutional neural network.

neuroscience↗

bioRxiv2019

Choosing to view morbid information involves reward circuitry

People often seek out stories, videos or images that detail death, violence or harm. Considering the ubiquity of this behavior, it is surprising that we know very little about the neural circuits involved in choosing negative information. Here we show that choosing intensely negative stimuli engages similar brain regions as those that support extrinsic incentives and \"regular\" curiosity. Participants made choices to view negative and positive images, based on negative (e.g., a soldier kicks a civilian against his head) and positive (e.g., children throw flower petals at a wedding) verbal cues. We hypothesized that the conflicting, but relatively informative act of choosing to view a negative image, resulted in stronger activation of reward circuitry as opposed to the relatively uncomplicated act of choosing to view a positive stimulus. Indeed, as preregistered, we found that choosing negative cues was associated with activation of the striatum, inferior frontal gyrus, anterior insula, and anterior cingulate cortex, both when contrasting against a passive viewing condition, and when contrasting against positive cues. These findings nuance models of decision-making, valuation and curiosity, and are an important starting point when considering the value of seeking out negative content.

neuroscience↗

bioRxiv2018

Scene complexity modulates degree of feedback activity during object recognition in natural scenes

Object recognition is thought to be mediated by rapid feed-forward activation of object-selective cortex, with limited contribution of feedback. However, disruption of visual evoked activity beyond feed-forward processing stages has been demonstrated to affect object recognition performance. Here, we unite these findings by reporting that the detection of target objects in natural scenes is selectively characterized by enhanced feedback when these objects are embedded in high complexity scenes. Human participants performed an animal target detection task on scenes with low, medium or high complexity as determined by a biologically plausible computational model of low-level contrast statistics. Three converging lines of evidence indicate that feedback was enhanced during categorization of scenes with high, but not low or medium complexity. First, functional magnetic resonance imaging (fMRI) activity in early visual cortex (V1) was selectively enhanced for target objects in scenes with high complexity. Second, event-related potentials (ERPs) evoked by high complexity scenes were selectively enhanced from 220 ms after stimulus-onset. Third, behavioral performance deteriorated for highly complex scenes when participants were pressed for time, but not when they could process the scenes fully and thereby benefit from the enhanced feedback. Formal modeling of the reaction time distributions revealed that object information accumulated more slowly for high complexity scenes (resulting in more errors especially for fast decisions), and directly related to the build-up of the feedback activity that was observed exclusively for high complexity scenes. Together, these results suggest that while feed-forward activity may suffice for simple scenes, the brain employs recurrent processing more adaptively in naturalistic settings, using minimal feedback for sparse, coherent scenes and increasing feedback for complex, fragmented scenes.\n\nAuthor summaryHow much neural processing is required to detect objects of interest in natural scenes? The astonishing speed of object recognition suggests that fast feed-forward buildup of perceptual activity is sufficient. However, this view is contradicted by findings that show that disruption of slower neural feedback leads to decreased detection performance. Our study unites these discrepancies by identifying scene complexity as a critical driver of neural feedback. We show how feedback is enhanced for complex, cluttered scenes compared to simple, well-organized scenes. Moreover, for complex scenes, more feedback is associated with better performances. These findings relate the flexibility of neural processes to perceptual decision-making by demonstrating that the brain dynamically directs neural resources based on the complexity of real-world visual inputs.

neuroscience↗