Biology subjects

Bonner, M. F.

Publications and source records attributed to Bonner, M. F..

8 recordsLinked to original sources

bioRxiv2026

Multidimensional dynamics of object representations in the human visual system

AO_SCPLOWBSTRACTC_SCPLOWNatural image representations are distributed across many dimensions of visual cortex activity, but little is known about how the multidimensional structure of these representations evolves over time following stimulus onset. Here we examined the temporal dynamics and latent dimensional structure of natural object representations in large-scale EEG and MEG data. We also compared these data with leading representational models derived from large-scale human similarity judgments and deep neural networks. Our findings reveal a rapid expansion of stimulus dimensionality in the brain, which peaks within 100 milliseconds and gradually decays over hundreds of milliseconds. The dynamics of these dimensionality changes tracked the decoding accuracy for both behavioral embeddings and neural network features, suggesting that dimensionality may be a general indicator of representational expressivity. Interestingly, the dimensionality of the neural representations could not be fully explained by leading behavior-based or neural network models. Follow-up experiments showed that the remaining neural variance carried additional perceptually relevant information not yet explained by leading models. Together, these findings reveal previously unrecognized complexity in measurements of dynamic human brain responses to natural objects.

neuroscience↗

bioRxiv2024

Convolutional architectures are cortex-aligned de novo

What underlies the emergence of cortex-aligned representations in deep neural network models of vision? Earlier work suggested that shared architectural constraints were a major factor, but the success of widely varied architectures after pre-training raises critical questions about the importance of architectural constraints. Here we show that in wide networks with minimal training, architectural inductive biases have a prominent role. We examined networks with varied architectures but no pre-training and quantified their ability to predict image representations in the visual cortices of monkeys and humans. We found that cortex-aligned representations emerge in convolutional architectures that combine two key manipulations of dimensionality: compression in the spatial domain, through pooling, and expansion in the feature domain by increasing the number of channels. We further show that the inductive biases of convolutional architectures are critical for obtaining performance gains from feature expansion--dimensionality manipulations were relatively ineffective in other architectures and in convolutional models with targeted lesions. Our findings suggest that the architectural constraints of convolutional networks are sufficiently close to the constraints of biological vision to allow many aspects of cortical visual representation to emerge even before synaptic connections have been tuned through experience.

animal behavior and cognition↗

bioRxiv2022

High-performing neural network models of visual cortex benefit from high latent dimensionality

Geometric descriptions of deep neural networks (DNNs) have the potential to uncover core representational principles of computational models in neuroscience. Here we examined the geometry of DNN models of visual cortex by quantifying the latent dimensionality of their natural image representations. A popular view holds that optimal DNNs compress their representations onto low-dimensional subspaces to achieve invariance and robustness, which suggests that better models of visual cortex should have lower dimensional geometries. Surprisingly, we found a strong trend in the opposite direction--neural networks with high-dimensional image subspaces tended to have better generalization performance when predicting cortical responses to held-out stimuli in both monkey electrophysiology and human fMRI data. Moreover, we found that high dimensionality was associated with better performance when learning new categories of stimuli, suggesting that higher dimensional representations are better suited to generalize beyond their training domains. These findings suggest a general principle whereby high-dimensional geometry confers computational benefits to DNN models of visual cortex.

neuroscience↗

bioRxiv2021

Tuning in scene-preferring cortex for mid-level visual features gives rise to selectivity across multiple levels of stimulus complexity

The scene-preferring portion of the human ventral visual stream, known as the parahippocampal place area (PPA), responds to scenes and landmark objects, which tend to be large in real-world size, fixed in location, and inanimate. However, the PPA also exhibits preferences for low-level contour statistics, including rectilinearity and cardinal orientations, that are not directly predicted by theories of scene- and landmark-selectivity. It is unknown whether these divergent findings of both low- and high-level selectivity in the PPA can be explained by a unified computational theory. To address this issue, we fit feedforward computational models of visual feature coding to the image-evoked fMRI responses of the PPA, and we performed a series of high-throughput experiments on these models. Our findings show that feedforward models of the PPA exhibit emergent selectivity across multiple levels of complexity, giving rise to seemingly high-level preferences for scenes and for objects that are large, spatially fixed, and inanimate/manmade while simultaneously yielding low-level preferences for rectilinear shapes and cardinal orientations. These results reconcile disparate theories of PPA function in a unified model of feedforward feature coding, and they demonstrate how multifaceted selectivity profiles naturally emerge from the feedforward computations of visual cortex and the natural statistics of images. SIGNIFICANCE STATEMENTVisual neuroscientists characterize cortical selectivity by identifying stimuli that drive regional responses. A perplexing finding is that many higher-order visual regions exhibit selectivity profiles spanning multiple levels of complexity: they respond to highly complex categories, such as scenes and landmarks, but also to surprisingly simplistic features, such as specific contour orientations. Using large-scale computational analyses and human brain imaging, we show how multifaceted selectivity in scene-preferring cortex can emerge from the feedforward, hierarchical coding of visual features. Our work reconciles seemingly divergent findings of selectivity in scene-preferring cortex and suggests that surprisingly simple feedforward feature representations may be central to the category-selective organization of the human visual system.

neuroscience↗

bioRxiv2020

Unveiling functions of the visual cortex using task-specific deep neural networks

The human visual cortex enables visual perception through a cascade of hierarchical computations in cortical regions with distinct functionalities. Here, we introduce an AI-driven approach to discover the functional mapping of the visual cortex. We related human brain responses to scene images measured with functional MRI (fMRI) systematically to a diverse set of deep neural networks (DNNs) optimized to perform different scene perception tasks. We found a structured mapping between DNN tasks and brain regions along the ventral and dorsal visual streams. Low-level visual tasks mapped onto early brain regions, 3-dimensional scene perception tasks mapped onto the dorsal stream, and semantic tasks mapped onto the ventral stream. This mapping was of high fidelity, with more than 60% of the explainable variance in nine key regions being explained. Together, our results provide a novel functional mapping of the human visual cortex and demonstrate the power of the computational approach.

neuroscience↗

bioRxiv2020

Object representations in the human brain reflect the co-occurrence statistics of vision and language

A central regularity of visual perception is the co-occurrence of objects in the natural environment. Here we use machine learning and fMRI to test the hypothesis that object co-occurrence statistics are encoded in the human visual system and elicited by the perception of individual objects. We identified low-dimensional representations that capture the latent statistical structure of object co-occurrence in real-world scenes, and we mapped these statistical representations onto voxelwise fMRI responses during object viewing. We found that cortical responses to single objects were predicted by the statistical ensembles in which they typically occur, and that this link between objects and their visual contexts was made most strongly in the anterior portion of the scene-selective parahippocampal place area. In contrast, a language-based statistical model of the co-occurrence of object names in written text predicted responses in neighboring regions of object-selective visual cortex. Together, these findings show that the sensory coding of objects in the human brain reflects the latent statistics of object context in visual and linguistic experience.

neuroscience↗

bioRxiv2017

Neural coding of fine-grained object knowledge in perirhinal cortex

SummaryOver 40 years of research has examined the role of the ventral visual stream in transforming retinal inputs into high-level representations of object identity [1-6]. However, there remains an ongoing debate over the role of the ventral stream in coding abstract semantic content, which relies on stored knowledge, versus perceptual content that relies only on retinal inputs [7-12]. A major difficulty in adjudicating between these mechanisms is that the semantic similarity of objects is often highly confounded with their perceptual similarity (e.g., animate things are more perceptually similar to other animate things than to inanimate things). To address this problem, we developed a paradigm that exploits the statistical regularities of object colors while perfectly controlling for perceptual shape information, allowing us to dissociate lower-level perceptual features (i.e., color perception) from higher-level semantic knowledge (i.e., color meaning). Using multivoxel-pattern analyses of fMRI data, we observed a striking double dissociation between the processing of color information at a perceptual and at a semantic level along the posterior to anterior axis of the ventral visual pathway. Specifically, we found that the visual association region V4 assigned similar representations to objects with similar colors, regardless of object category. In contrast, perirhinal cortex, at the apex of the ventral visual stream, assigned similar representations to semantically similar objects, even when this was in opposition to their perceptual similarity. These findings suggest that perirhinal cortex untangles the representational space of lower-level perceptual features and organizes visual objects according to their semantic interpretations.

neuroscience↗

bioRxiv2017

Computational mechanisms underlying cortical responses to the affordance properties of visual scenes

Biologically inspired deep convolutional neural networks (CNNs), trained for computer vision tasks, have been found to predict cortical responses with remarkable accuracy. However, the complex internal operations of these models remain poorly understood, and the factors that account for their success are unknown. Here we developed a set of techniques for using CNNs to gain insights into the computational mechanisms underlying cortical responses. We focused on responses in the occipital place area (OPA), a scene-selective region of dorsal occipitoparietal cortex. In a previous study, we showed that fMRI activation patterns in the OPA contain information about the navigational affordances of scenes: that is, information about where one can and cannot move within the immediate environment. We hypothesized that this affordance information could be extracted using a set of purely feedforward computations. To test this idea, we examined a deep CNN with a feedforward architecture that had been previously trained for scene classification. We found that the CNN was highly predictive of OPA representations, and, importantly, that it accounted for the portion of OPA variance that reflected the navigational affordances of scenes. The CNN could thus serve as an image-computable candidate model of affordance-related responses in the OPA. We then ran a series of in silico experiments on this model to gain insights into its internal computations. These analyses showed that the computation of affordance-related features relied heavily on visual information at high-spatial frequencies and cardinal orientations, both of which have previously been identified as low-level stimulus preferences of scene-selective visual cortex. These computations also exhibited a strong preference for information in the lower visual field, which is consistent with known retinotopic biases in the OPA. Visualizations of feature selectivity within the CNN suggested that affordance-based responses encoded features that define the layout of the spatial environment, such as boundary-defining junctions and large extended surfaces. Together, these results map the sensory functions of the OPA onto a fully quantitative model that provides insights into its visual computations. More broadly, they advance integrative techniques for understanding visual cortex across multiple level of analysis: from the identification of cortical sensory functions to the modeling of their underlying algorithmic implementations.\n\nAUTHOR SUMMARYHow does visual cortex compute behaviorally relevant properties of the local environment from sensory inputs? For decades, computational models have been able to explain only the earliest stages of biological vision, but recent advances in the engineering of deep neural networks have yielded a breakthrough in the modeling of high-level visual cortex. However, these models are not explicitly designed for testing neurobiological theories, and, like the brain itself, their complex internal operations remain poorly understood. Here we examined a deep neural network for insights into the cortical representation of the navigational affordances of visual scenes. In doing so, we developed a set of high-throughput techniques and statistical tools that are broadly useful for relating the internal operations of neural networks with the information processes of the brain. Our findings demonstrate that a deep neural network with purely feedforward computations can account for the processing of navigational layout in high-level visual cortex. We next performed a series of experiments and visualization analyses on this neural network, which characterized a set of stimulus input features that may be critical for computing navigationally related cortical representations and identified a set of high-level, complex scene features that may serve as a basis set for the cortical coding of navigational layout. These findings suggest a computational mechanism through which high-level visual cortex might encode the spatial structure of the local navigational environment, and they demonstrate an experimental approach for leveraging the power of deep neural networks to understand the visual computations of the brain.

neuroscience↗