Biology subjects

Issa, E. B.

Publications and source records attributed to Issa, E. B..

8 recordsLinked to original sources

bioRxiv2026

Efficient task generalization and humanlike face perception in models that learn to discriminate face geometry

Artificial deep neural networks (DNNs) can excel at face recognition from 2D photographs where both shape and appearance cues abound; however, DNNs have rarely been challenged to recognize faces strictly based on face geometry. Here, we show that DNNs, even those fine-tuned on face photographs, had almost no generalization performance to a new geometry-based face task, while in the opposite direction, networks fine-tuned only on geometrically defined, textureless faces readily generalized to textured faces. To learn geometry in a more practical setting with colored and textured faces, we trained discrimination on face emotion in addition to face identity, which resulted in less texture bias and generalized well across face tasks. Learning in this way from just four individuals and their expressions generalized to unseen individuals, even exceeding standard models which are trained on classifying hundreds of face identities. Compared to standard models, emotion and identity trained models developed more humanlike errors in the identities or emotions that they confused. This novel method learns in a humanlike manner using only a few individuals but enriched with expressions that widely vary face geometry - similar to early human experience during child-parent interactions. Thus, this bioinspired work has broad implications for how moving toward humanlike learning of geometry in artificial vision can be both highly sample efficient and highly performing.

neuroscience↗

bioRxiv2024

Humans and marmosets share similar face recognition signatures in shape-based visual face discrimination behavior

Our ability to identify faces is thought to depend on high-level visual processing in the brain. Nonetheless, studies of face recognition have generally relied on 2D face photographs where low-level strategies relying on texture and appearance cues can be employed to adequately support high face identification performance. Here, we designed a fine face discrimination task under 3D pose and lighting variation that was purely based on shape, a task which challenged state-of-the-art artificial vision systems compared to object recognition tasks. In contrast, humans performed this shape based face task at comparable levels to their object recognition performance. We then tested one of the smallest simian primates on this human-level, machine-difficult visual task, the common marmoset - a small, New World monkey. Marmosets successfully discriminated between face identities across 3D viewing conditions based purely on face shape. Their face recognition performance was on par with their object recognition performance and exhibited face-specific behavioral signatures similar to humans, including lower performance for inverted faces, faces lit from below, and contrast reversed faces. These results demonstrate that a high-level visual behavior, invariant face recognition based purely on geometry and not additional texture and appearance cues, is shared across simian primates from among the smallest to the most advanced, consistent with the presence of common underlying high-level visual brain areas across simian primates.

neuroscience↗

bioRxiv2023

Factorized visual representations in the primate visual system and deep neural networks

Object classification has been proposed as a principal objective of the primate ventral visual stream and has been used as an optimization target for deep neural network models (DNNs) of the visual system. However, visual brain areas represent many different types of information, and optimizing for classification of object identity alone does not constrain how other information may be encoded in visual representations. Information about different scene parameters may be discarded altogether ("invariance"), represented in non-interfering subspaces of population activity ("factorization") or encoded in an entangled fashion. In this work, we provide evidence that factorization is a normative principle of biological visual representations. In the monkey ventral visual hierarchy, we found that factorization of object pose and background information from object identity increased in higher-level regions and strongly contributed to improving object identity decoding performance. We then conducted a large-scale analysis of factorization of individual scene parameters - lighting, background, camera viewpoint, and object pose - in a diverse library of DNN models of the visual system. Models which best matched neural, fMRI and behavioral data from both monkeys and humans across 12 datasets tended to be those which factorized scene parameters most strongly. Notably, invariance to these parameters was not as consistently associated with matches to neural and behavioral data, suggesting that maintaining non-class information in factorized activity subspaces is often preferred to dropping it altogether. Thus, we propose that factorization of visual scene information is a widely used strategy in brains and DNN models thereof.

neuroscience↗

bioRxiv2020

Conserved core visual object recognition across simian primates: Marmoset image-by-image behavior mirrors that of humans and macaques

Bigger brains are thought to support richer abilities, including perceptual abilities. But bigger brains are typically organized differently (e.g., with more cortical areas). Thus, the extent to which a neural systems size versus organization underlies complex abilities remains unclear. The marmoset monkey is evolutionarily peculiar: it has a small brain, yet many cortical areas. We used this natural experiment to test organization as source of high-level visual abilities independent of size, via large-scale psychophysics comparing marmosets to different species on identical tasks. Marmosets far out--performed rats--a marmoset-sized rodent--on a simple visual recognition task. On another visual task, which is difficult for both humans and machines, marmosets achieved high performance. Strikingly, their image-by-image behavior revealed that they did so in a manner highly similar to humans--marmosets were nearly as human-like as were macaques. These results suggest a key role for brain organization--not simply size--in the evolution of sophisticated abilities.

neuroscience↗

bioRxiv2018

Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?

The internal representations of early deep artificial neural networks (ANNs) were found to be remarkably similar to the internal neural representations measured experimentally in the primate brain. Here we ask, as deep ANNs have continued to evolve, are they becoming more or less brain-like? ANNs that are most functionally similar to the brain will contain mechanisms that are most like those used by the brain. We therefore developed Brain-Score - a composite of multiple neural and behavioral benchmarks that score any ANN on how similar it is to the brains mechanisms for core object recognition - and we deployed it to evaluate a wide range of state-of-the-art deep ANNs. Using this scoring system, we here report that: (1) DenseNet-169, CORnet-S and ResNet-101 are the most brain-like ANNs. (2) There remains considerable variability in neural and behavioral responses that is not predicted by any ANN, suggesting that no ANN model has yet captured all the relevant mechanisms. (3) Extending prior work, we found that gains in ANN ImageNet performance led to gains on Brain-Score. However, correlation weakened at [≥] 70% top-1 ImageNet performance, suggesting that additional guidance from neuroscience is needed to make further advances in capturing brain mechanisms. (4) We uncovered smaller (i.e. less complex) ANNs that are more brain-like than many of the best-performing ImageNet models, which suggests the opportunity to simplify ANNs to better understand the ventral stream. The scoring system used here is far from complete. However, we propose that evaluating and tracking model-benchmark correspondences through a Brain-Score that is regularly updated with new brain data is an exciting opportunity: experimental benchmarks can be used to guide machine network evolution, and machine networks are mechanistic hypotheses of the brains network and thus drive next experiments. To facilitate both of these, we release Brain-Score.org: a platform that hosts the neural and behavioral benchmarks, where ANNs for visual processing can be submitted to receive a Brain-Score and their rank relative to other models, and where new experimental data can be naturally incorporated.

neuroscience↗

bioRxiv2018

Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior

Non-recurrent deep convolutional neural networks (DCNNs) are currently the best models of core object recognition; a behavior supported by the densely recurrent primate ventral stream, culminating in the inferior temporal (IT) cortex. Are these recurrent circuits critical to ventral streams execution of this behavior? We reasoned that, if recurrence is critical, then primates should outperform feedforward-only DCNNs for some images, and that these images should require additional processing time beyond the feedforward IT response. Here we first used behavioral methods to discover hundreds of these \"challenge\" images. Second, using large-scale IT electrophysiology in animals performing core recognition tasks, we observed that behaviorally-sufficient, linearly-decodable object identity solutions emerged ~30ms (on average) later in IT for challenge images compared to DCNN and primate performance-matched \"control\" images. We observed these same late solutions even during passive viewing. Third, consistent with a failure of feedforward computations, the behaviorally-critical late-phase IT population response patterns evoked by the challenge images were poorly predicted by DCNN activations. Interestingly, deeper CNNs better predicted these late IT responses, suggesting a functional equivalence between recurrence and additional nonlinear transformations. Our results argue that automatically-evoked recurrent circuits are critical even for rapid object identification. By precisely comparing current DCNNs, primate behavior and IT population dynamics, we provide guidance for future recurrent model development.

neuroscience↗

bioRxiv2018

Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks

Primates--including humans--can typically recognize objects in visual images at a glance even in the face of naturally occurring identity-preserving image transformations (e.g. changes in viewpoint). A primary neuroscience goal is to uncover neuron-level mechanistic models that quantitatively explain this behavior by predicting primate performance for each and every image. Here, we applied this stringent behavioral prediction test to the leading mechanistic models of primate vision (specifically, deep, convolutional, artificial neural networks; ANNs) by directly comparing their behavioral signatures against those of humans and rhesus macaque monkeys. Using high-throughput data collection systems for human and monkey psychophysics, we collected over one million behavioral trials for 2400 images over 276 binary object discrimination tasks. Consistent with previous work, we observed that state-of-the-art deep, feed-forward convolutional ANNs trained for visual categorization (termed DCNNIC models) accurately predicted primate patterns of object-level confusion. However, when we examined behavioral performance for individual images within each object discrimination task, we found that all tested DCNNIC models were significantly non-predictive of primate performance, and that this prediction failure was not accounted for by simple image attributes, nor rescued by simple model modifications. These results show that current DCNNIC models cannot account for the image-level behavioral patterns of primates, and that new ANN models are needed to more precisely capture the neural mechanisms underlying primate object vision. To this end, large-scale, high-resolution primate behavioral benchmarks--such as those obtained here--could serve as direct guides for discovering such models.\n\nSIGNIFICANCE STATEMENTRecently, specific feed-forward deep convolutional artificial neural networks (ANNs) models have dramatically advanced our quantitative understanding of the neural mechanisms underlying primate core object recognition. In this work, we tested the limits of those ANNs by systematically comparing the behavioral responses of these models with the behavioral responses of humans and monkeys, at the resolution of individual images. Using these high-resolution metrics, we found that all tested ANN models significantly diverged from primate behavior. Going forward, these high-resolution, large-scale primate behavioral benchmarks could serve as direct guides for discovering better ANN models of the primate visual system.

neuroscience↗

bioRxiv2016

Evidence that the ventral stream codes the errors used in hierarchical inference and learning

Ventral visual stream neural responses are dynamic, even for static image presentations. However, dynamical neural models of visual cortex are lacking as most progress has been made modeling static, time-averaged responses. Here, we studied population neural dynamics during face detection across three cortical processing stages. Remarkably, ~30 milliseconds after the initially evoked response, we found that neurons in intermediate level areas decreased their preference for faces, becoming anti-face preferring on average even while neurons in higher level areas achieved and maintained a face preference. This pattern of hierarchical neural dynamics was inconsistent with extensions of standard feedforward circuits that implemented recurrence within a cortical stage. Rather, recurrent models computing errors between stages captured the observed temporal signatures. Without additional parameter fitting, this model of neural dynamics, which simply augments the standard feedforward model of online vision to encode errors, also explained seemingly disparate dynamical phenomena in the ventral stream.

neuroscience↗