Biology subjects
Publications and source records attributed to Rajalingham, R..
Cognitive neuroscience posits that humans perform physical inferences using mental simulations. Here, we test this hypothesis by analyzing neural activity in the monkeys frontal cortex in a ball interception task. We find a low-dimensional neural embedding of the ball position that tracks the ball both when it is visible and invisible. This embedding may serve as a neural substrate for mental simulation.
Primates can richly parse sensory inputs to infer latent information, and adjust their behavior accordingly. It has been hypothesized that such flexible inferences are aided by simulations of internal models of the external world. However, evidence supporting this hypothesis has been based on behavioral models that do not emulate neural computations. Here, we test this hypothesis by directly comparing the behavior of humans and monkeys in a ball interception task to that of recurrent neural network (RNN) models with or without the capacity to "simulate" the underlying latent variables. Humans and monkeys had strikingly similar behavioral patterns suggesting common underlying neural computations. Comparison between primates and a large class of RNNs revealed that only RNNs that were optimized to simulate the position of the ball were able to accurately capture key features of the behavior such as systematic biases in the inference process. These results are consistent with the hypothesis that primates use mental simulation to make flexible inferences. Moreover, our work highlights a general strategy for using model neural systems to test computational hypotheses of higher brain function.
Challenges in behavioral optogenetics in large brains demand development of a chronically implantable platform for light delivery. We have developed Opto-Array, a chronically implantable array of LEDs for high-throughput optogenetic perturbation in non-human primates. We tested the Opto-Array in the primary visual cortex of a macaque monkey, and demonstrated that optogenetic cortical silencing by the Opto-Array results in reliable retinotopic visual deficits on a luminance discrimination task.
Skilled human readers can readily recognize written letters and letter strings. This domain of visual recognition, known as orthographic processing, is foundational to human reading, but it is unclear how it is supported by neural populations in the human brain. Behavioral research has shown that non-human primates (baboons) can learn to distinguish written English words from pseudo-words (lexical decision), successfully generalize that behavior to novel strings, and exhibit behavioral error patterns that are consistent with humans. Thus, non-human primate models, while not capturing the entirety of human reading abilities, may provide a unique opportunity to investigate the neuronal mechanisms underlying orthographic processing. Here, we investigated the neuronal representation of letters and letter strings in the ventral visual stream of naive macaque monkeys, and asked to what extent these representations could support visual word recognition. We recorded the activity of hundreds of neurons at the top two levels of the ventral visual form processing pathway (V4 and IT) while monkeys passively viewed images of letters, English words, and non-word letter strings. Linear decoders were used to probe whether those neural responses could support a battery of orthographic processing tasks such as invariant letter identification and lexical decision. We found that IT-based decoders achieved baboon-level performance on these tasks, with a pattern of errors highly correlated to the previously reported primate behavior. This capacity to support orthographic processing tasks was also present in the high-layer units of state-of-the-art artificial neural network models of the ventral stream, but not in the low-layer representations of those models. Taken together, these results show that the IT cortex of untrained monkeys contains a reservoir of precursor features from which downstream brain regions could, with some supervised instruction, learn to support the visual recognition of written words. This suggests that the acquisition of reading in humans did not require a full rebuild of visual processing, but rather the recycling of a brain network evolved for other visual functions.
The internal representations of early deep artificial neural networks (ANNs) were found to be remarkably similar to the internal neural representations measured experimentally in the primate brain. Here we ask, as deep ANNs have continued to evolve, are they becoming more or less brain-like? ANNs that are most functionally similar to the brain will contain mechanisms that are most like those used by the brain. We therefore developed Brain-Score - a composite of multiple neural and behavioral benchmarks that score any ANN on how similar it is to the brains mechanisms for core object recognition - and we deployed it to evaluate a wide range of state-of-the-art deep ANNs. Using this scoring system, we here report that: (1) DenseNet-169, CORnet-S and ResNet-101 are the most brain-like ANNs. (2) There remains considerable variability in neural and behavioral responses that is not predicted by any ANN, suggesting that no ANN model has yet captured all the relevant mechanisms. (3) Extending prior work, we found that gains in ANN ImageNet performance led to gains on Brain-Score. However, correlation weakened at [≥] 70% top-1 ImageNet performance, suggesting that additional guidance from neuroscience is needed to make further advances in capturing brain mechanisms. (4) We uncovered smaller (i.e. less complex) ANNs that are more brain-like than many of the best-performing ImageNet models, which suggests the opportunity to simplify ANNs to better understand the ventral stream. The scoring system used here is far from complete. However, we propose that evaluating and tracking model-benchmark correspondences through a Brain-Score that is regularly updated with new brain data is an exciting opportunity: experimental benchmarks can be used to guide machine network evolution, and machine networks are mechanistic hypotheses of the brains network and thus drive next experiments. To facilitate both of these, we release Brain-Score.org: a platform that hosts the neural and behavioral benchmarks, where ANNs for visual processing can be submitted to receive a Brain-Score and their rank relative to other models, and where new experimental data can be naturally incorporated.
Extensive research suggests that the inferior temporal (IT) population supports visual object recognition behavior. However, causal evidence for this hypothesis has been equivocal, particularly beyond the specific case of face-selective sub-regions of IT. Here, we directly tested this hypothesis by pharmacologically inactivating individual, millimeter-scale sub-regions of IT while monkeys performed several object discrimination tasks, interleaved trial-by-trial. First, we observed that IT inactivation resulted in reliable contralateral-biased task-selective behavioral deficits. Moreover, inactivating different IT sub-regions resulted in different patterns of task deficits, each predicted by that sub-regions neuronal object discriminability. Finally, the similarity between different inactivation effects was tightly related to the anatomical distance between corresponding inactivation sites. Taken together, these results provide direct evidence that IT cortex causally supports general core object recognition, and that the underlying IT codes are topographically organized.
Primates--including humans--can typically recognize objects in visual images at a glance even in the face of naturally occurring identity-preserving image transformations (e.g. changes in viewpoint). A primary neuroscience goal is to uncover neuron-level mechanistic models that quantitatively explain this behavior by predicting primate performance for each and every image. Here, we applied this stringent behavioral prediction test to the leading mechanistic models of primate vision (specifically, deep, convolutional, artificial neural networks; ANNs) by directly comparing their behavioral signatures against those of humans and rhesus macaque monkeys. Using high-throughput data collection systems for human and monkey psychophysics, we collected over one million behavioral trials for 2400 images over 276 binary object discrimination tasks. Consistent with previous work, we observed that state-of-the-art deep, feed-forward convolutional ANNs trained for visual categorization (termed DCNNIC models) accurately predicted primate patterns of object-level confusion. However, when we examined behavioral performance for individual images within each object discrimination task, we found that all tested DCNNIC models were significantly non-predictive of primate performance, and that this prediction failure was not accounted for by simple image attributes, nor rescued by simple model modifications. These results show that current DCNNIC models cannot account for the image-level behavioral patterns of primates, and that new ANN models are needed to more precisely capture the neural mechanisms underlying primate object vision. To this end, large-scale, high-resolution primate behavioral benchmarks--such as those obtained here--could serve as direct guides for discovering such models.\n\nSIGNIFICANCE STATEMENTRecently, specific feed-forward deep convolutional artificial neural networks (ANNs) models have dramatically advanced our quantitative understanding of the neural mechanisms underlying primate core object recognition. In this work, we tested the limits of those ANNs by systematically comparing the behavioral responses of these models with the behavioral responses of humans and monkeys, at the resolution of individual images. Using these high-resolution metrics, we found that all tested ANN models significantly diverged from primate behavior. Going forward, these high-resolution, large-scale primate behavioral benchmarks could serve as direct guides for discovering better ANN models of the primate visual system.