Biology subjects

Margalit, E.

Publications and source records attributed to Margalit, E..

6 recordsLinked to original sources

bioRxiv2024

Do Topographic Deep ANN Models of the Primate Ventral Stream Predict the Perceptual Effects of Direct IT Cortical Interventions?

Ever-advancing artificial neural network (ANN) models of the ventral visual stream capture core object recognition behavior and the neural mechanisms underlying it with increasing precision. These models take images as input, propagate through simulated neural representations that resemble biological neural representations at all stages of the primate ventral stream, and produce simulated behavioral choices that resemble primate behavioral choices. We here extend this modeling approach to make and test predictions of neural intervention experiments. Specifically, we enable a new prediction regime for topographic deep ANN (TDANN) models of primate visual processing through the development of perturbation modules that translate micro-stimulation, optogenetic suppression, and muscimol suppression into changes in model neural activity. This unlocks the ability to predict the behavioral effects from particular neural perturbations. We compare these predictions with the key results from the primate IT perturbation experimental literature via a suite of nine corresponding benchmarks. Without any fitting to the benchmarks, we find that TDANN models generated via co-training with both a spatial correlation loss and a standard categorization task qualitatively predict all nine behavioral results. In contrast, TDANN models generated via random topography or via topographic unit arrangement after classification training predict less than half of those results. However, the models quantitative predictions are consistently misaligned with experimental data, over-predicting the magnitude of some behavioral effects and under-predicting others. None of the TDANN models were built with separate model hemispheres and thus, unsurprisingly, all fail to predict hemispheric-dependent effects. Taken together, these findings indicate that current topographic deep ANN models paired with perturbation modules are reasonable guides to predict the qualitative results of direct causal experiments in IT, but that improved TDANN models will be needed for precise quantitative predictions.

neuroscience↗

bioRxiv2023

A single computational objective drives specialization of streams in visual cortex

Human visual cortex is organized into dorsal, lateral, and ventral streams. A long-standing hypothesis is that the functional organization into streams emerged to support distinct visual behaviors. Here, we compare neural network-based computational models against a massive fMRI dataset to investigate why visual streams emerge. We find that a self-supervised Topographic Deep Artificial Neural Network (TDANN), which encourages nearby units to respond similarly, better captures brain responses, as well as spatial segregation and functional differentiation across streams, than DANN models trained for stream-specific visual behaviors. These findings challenge the prevailing view that streams evolved to separately support different behaviors and suggest instead the possibility that functional organization can arise from a single principle: learning generally useful visual representations subject to local spatial constraints.

neuroscience↗

bioRxiv2023

A Unifying Principle for the Functional Organization of Visual Cortex

A key feature of many cortical systems is functional organization: the arrangement of neurons with specific functional properties in characteristic spatial patterns across the cortical surface. However, the principles underlying the emergence and utility of functional organization are poorly understood. Here we develop the Topographic Deep Artificial Neural Network (TDANN), the first unified model to accurately predict the functional organization of multiple cortical areas in the primate visual system. We analyze the key factors responsible for the TDANNs success and find that it strikes a balance between two specific objectives: achieving a task-general sensory representation that is self-supervised, and maximizing the smoothness of responses across the cortical sheet according to a metric that scales relative to cortical surface area. In turn, the representations learned by the TDANN are lower dimensional and more brain-like than those in models that lack a spatial smoothness constraint. Finally, we provide evidence that the TDANNs functional organization balances performance with inter-area connection length, and use the resulting models for a proof-of-principle optimization of cortical prosthetic design. Our results thus offer a unified principle for understanding functional organization and a novel view of the functional role of the visual system in particular.

neuroscience↗

bioRxiv2021

Increasing neural network robustness improves match to macaque V1 eigenspectrum, spatial frequency preference and predictivity

Task-optimized convolutional neural networks (CNNs) show striking similarities to the ventral visual stream. However, human-imperceptible image perturbations can cause a CNN to make incorrect predictions. Here we provide insight into this brittleness by investigating the representations of models that are either robust or not robust to image perturbations. Theory suggests that the robustness of a system to these perturbations could be related to the power law exponent of the eigenspectrum of its set of neural responses, where power law exponents closer to and larger than one would indicate a system that is less susceptible to input perturbations. We show that neural responses in mouse and macaque primary visual cortex (V1) obey the predictions of this theory, where their eigenspectra have power law exponents of at least one. We also find that the eigenspectra of model representations decay slowly relative to those observed in neurophysiology and that robust models have eigenspectra that decay slightly faster and have higher power law exponents than those of non-robust models. The slow decay of the eigenspectra suggests that substantial variance in the model responses is related to the encoding of fine stimulus features. We therefore investigated the spatial frequency tuning of artificial neurons and found that a large proportion of them preferred high spatial frequencies and that robust models had preferred spatial frequency distributions more aligned with the measured spatial frequency distribution of macaque V1 cells. Furthermore, robust models were quantitatively better models of V1 than non-robust models. Our results are consistent with other findings that there is a misalignment between human and machine perception. They also suggest that it may be useful to penalize slow-decaying eigenspectra or to bias models to extract features of lower spatial frequencies during task-optimization in order to improve robustness and V1 neural response predictivity. Author summaryConvolutional neural networks (CNNs) are the most quantitatively accurate models of multiple visual areas. In contrast to humans, however, their image classification behaviour can be modified drastically by human-imperceptible image perturbations. To provide insight as to why CNNs are so brittle, we investigated the image features extracted by models that are robust and not robust to these image perturbations. We found that CNNs had a preference for high spatial frequency image features, unlike primary visual cortex (V1) cells. Models that were more robust to image perturbations had a preference for image features more aligned with those extracted by V1 and also improved predictions of neural responses in V1. This suggests that the dependence on high-frequency image features for image classification may be related to the image perturbations affecting models but not humans. Our work is consistent with other findings that CNNs may be relying on image features not aligned with those used by humans for image classification and suggests possible optimization targets to improve the robustness of and the V1 correspondence of CNNs.

neuroscience↗

bioRxiv2020

Extensive individual differences of category information in ventral temporal cortex in the congenitally blind

Human ventral temporal cortex (VTC) is a cortical expanse that performs different functions and computations, but is especially critical for visual categorization. Nevertheless, accumulating evidence shows that category-selective regions persist in VTC in the absence of visual experience - for example, in congenitally blind (CB) participants. Despite this evidence, a large body of previous work comparing functional representations in VTC between sighted and CB participants performed univariate analyses at the group level, which assume a homogeneous population - an assumption that has not been formally tested until the present study. Specifically, using fMRI in CB and sighted participants (male and female), we empirically show that at the group level, distributed category representations in VTC are more reliable in the sighted (when viewing visual stimuli) compared to the CB (when hearing auditorily-substituted visual stimuli). Despite these group differences, there is extensive heterogeneity in VTC category representations in the CB to the point that VTC category representations in a subset of CB participants (some who were born without eyes, but not all) are more similar to sighted individuals compared to other CB participants. Together, our findings support a novel idea that driving factors contributing to the formation of VTC category representations in the blind are subject-specific, which complements factors that may generalize across group members. More broadly, the present findings caution conclusions of homogeneity across subjects within a group when performing group neuroimaging analyses without explicitly quantifying individual differences.

neuroscience↗

bioRxiv2018

A critical assessment of data quality and venous effects in ultra-high-resolution fMRI

Advances in hardware, pulse sequences, and reconstruction techniques have made it possible to perform functional magnetic resonance imaging (fMRI) at sub-millimeter resolution while maintaining high spatial coverage and acceptable signal-to-noise ratio. Here, we examine whether ultra-high-resolution fMRI can be exploited for routine use in neuroscience research. We conducted fMRI in human visual cortex during a simple event-related visual experiment (7T, gradient-echo EPI, 0.8-mm isotropic voxels, 2.2-s sampling rate, 84 slices), and developed analysis and visualization tools to assess the quality of the data. We make three main observations. First, we find that the acquired fMRI images, combined with appropriate surface-based processing, provide reliable and accurate measurements of fine-scale blood oxygenation level dependent (BOLD) activity patterns. Second, we show that the highly folded structure of cortex causes substantial biases on spatial resolution and data visualization. Third, we examine the well-recognized issue of venous contributions to fMRI signals. In a systematic assessment of large sections of cortex measured at a fine scale, we show that time-averaged T2*-weighted EPI intensity is a simple, robust marker of venous effects. These venous effects are unevenly distributed across cortex, are more pronounced in gyri and outer cortical depths, and are, to a certain degree, in consistent locations across subjects relative to cortical folding. Furthermore, we show that these venous effects are strongly correlated with BOLD responses evoked by the experiment. We conclude that ultra-high-resolution fMRI can provide robust information about fine-scale BOLD activity patterns, but special care must be exercised in visualizing and interpreting these patterns, especially with regards to the confounding influence of the brains vasculature. To help translate these methodological findings to neuroscience research, we provide practical suggestions for both high-resolution and standard-resolution fMRI studies.

neuroscience↗