Biology subjects

Cottaris, N. P.

Publications and source records attributed to Cottaris, N. P..

12 recordsLinked to original sources

bioRxiv2026

Transformation-tolerant object recognition in tree shrews despite lacking a fovea

Object recognition depends on the ability to extract stable representations across changes in how they are viewed, yet it remains unclear how this capacity depends on visual acuity and cortical hierarchy. We combined behavioral testing and computational modeling to determine whether tree shrews, close relatives of primates with lower spatial acuity, can perform transformation-tolerant object recognition. Front-end modeling incorporating species-specific optics and photoreceptor sampling showed that, when scaled for acuity, tree shrew retinal filtering preserves the similarity structure of natural image categories relevant for object recognition. Behaviorally, tree shrews reliably discriminated complex objects across variations in position, scale, and viewpoint, including when embedded within natural scenes, and generalized to novel exemplars. Their recognition behavior was best explained by visual features emphasizing differences in global shape and size between objects and by representations from intermediate and deep layers of hierarchical neural network models. These results demonstrate that visual processing supporting object-level generalization can arise within visual systems lacking high-acuity front-end optics and establish the tree shrew as a key model for understanding the computational and evolutionary origins of high-level vision.

animal behavior and cognition↗

bioRxiv2025

An image-computable, spatio-chromatic RF model of the midget RGC mosaic across the retina

Accurate image-computable models of retinal ganglion cell (RGC) mosaics across the retina do not currently exist. Here, we deploy a novel computational frame-work which synthesizes mosaics of linear spatio-chromatic receptive fields (RFs) of ON midget RGCs (mRGCs) by integrating published anatomical, physiological, and optical quality measurements. We use the synthesized mRGC mosaics to simulate both in vivo and in vitro physiological experiments and demonstrate the models consistency with published data. The model enables computation of how visual performance is shaped by the representation of visual information provided by the linear spatiochromatic processing stage of midget RGCs. The developed computational framework carefully accounts for the effect of physiological optics on mRGC responses, enables comparison of in vivo and in vitro data, and allows exploration of how different assumptions about RF organization, such as selectivity for the type of cones pooled by the RF center mechanism, affect physiological responses and psychophysical performance. The open-source and freely available implementation provides a platform for understanding how the linear spatiochromatic receptive field representation of the mRGCs shapes visual performance, as well as a foundation for future work that incorporates response nonlinearities, temporal filtering, and extends to additional RGC mosaics.

neuroscience↗

bioRxiv2022

In vivo physiology of foveal retinal ganglion cells in Macaca fascicularis

The primate fovea is specialized for high acuity chromatic vision, with the highest density of cone photoreceptors and a disproportionately large representation in visual cortex. The unique visual properties conferred by the fovea are conveyed to the brain by retinal ganglion cells, the somas of which lie at the margin of the foveal pit. Microelectrode recordings of these centermost retinal ganglion cells have been challenging due to the fragility of the fovea in the excised retina. Here we overcome this challenge by combining high resolution fluorescence adaptive optics ophthalmoscopy with calcium imaging to optically record functional responses of foveal retinal ganglion cells in the living eye. We use this approach to study the chromatic responses and spatial transfer functions of retinal ganglion cells using spatially uniform fields modulated in different directions in color space and monochromatic drifting gratings. We recorded from over 350 cells across three Macaca fascicularis primates over a time period of weeks to months. We find that the majority of the L vs. M cone opponent cells serving the most central foveolar cones have spatial transfer functions that peak at high spatial frequencies (20-40 c/deg), reflecting strong surround inhibition that sacrifices sensitivity at low spatial frequencies but preserves the transmission of fine detail in the retinal image. In addition, we fit to the drifting grating data a detailed model of how ganglion cell responses draw on the cone mosaic to derive receptive field properties of L vs. M cone opponent cells at the very center of the foveola. The fits are consistent with the hypothesis that foveal midget ganglion cells are specialized to preserve information at the resolution of the cone mosaic. By characterizing the functional properties of retinal ganglion cells in vivo through adaptive optics, we characterize the response characteristics of these cells in situ.

neuroscience↗

bioRxiv2021

An Image Reconstruction Framework for Characterizing Early Vision

We developed an image-computable observer model of the initial visual encoding that operates on natural image input, based on the framework of Bayesian image reconstruction from the excitations of the retinal cone mosaic. Our model extends previous work on ideal observer analysis and evaluation of performance beyond psychophysical discrimination, takes into account the statistical regularities of the visual environment, and provides a unifying framework for answering a wide range of questions regarding the visual front end. Using the error in the reconstructions as a metric, we analyzed variations of the number of different photoreceptor types on human retina as an optimal design problem. In addition, the reconstructions allow both visualization and quantification of information loss due to physiological optics and cone mosaic sampling, and how these vary with eccentricity. Furthermore, in simulations of color deficiencies and interferometric experiments, we found that the reconstructed images provide a reasonable proxy for modeling subjects percepts. Lastly, we used the reconstruction-based observer for the analysis of psychophysical threshold, and found notable interactions between spatial frequency and chromatic direction in the resulting spatial contrast sensitivity function. Our method is widely applicable to experiments and applications in which the initial visual encoding plays an important role.

neuroscience↗

bioRxiv2019

A computational observer model of spatial contrast sensitivity: Effects of photocurrent encoding, fixational eye movements and inference engine

We have recently shown that using the information carried by the mosaic of cone excitations of a stationary retina, the relative spatial contrast sensitivity function (CSF) of a computational observer has the same shape as a typical human subject. Absolute human sensitivity, however, is lower than the computational observer by a factor of 5 to 10. Here we model how additional known features of early vision affect spatial contrast sensitivity: fixational eye movements and the conversion of cone photopigment excitations to cone photocurrent responses. For a computational observer that uses a linear classifier applied to the responses of a stimulus-matched linear filter, fixational eye movements substantially change the shape of the spatial CSF, primarily by reducing sensitivity at spatial frequencies above 10 c/deg. For a computational observer that uses a translation-invariant calculation, in which decisions are based on the squared response of a quadrature-pair of linear filters, the CSF shape is little changed by eye movements, but there is a two-fold reduction in sensitivity. The noise and response dynamics of conversion of cone excitations into photocurrent introduce an additional two-fold sensitivity decrease. Hence, the combined effects of fixational eye movements and phototransduction bring the absolute sensitivity of the translation-invariant computational observer CSF to within a factor of 1 to 2 of the human CSF. We note that the human CSF depends on processing of the initial representation by many thalamic and cortical neurons, which are individually quite noisy. Our computational modeling suggests that the net effect of this noise on contrast-detection performance, when considered at the neural population level and behavioral level, is quite small: the inference mechanisms that determine the CSF, presumably in cortex, make efficient use of the information available from the cone photocurrents of the fixating eye.

neuroscience↗

bioRxiv2019

Ray tracing 3D spectral scenes through human optics models

Scientists and engineers have created computations and made measurements that characterize the first steps of seeing. ISETBio software integrates such computations and data into an open-source software package. The initial ISETBio implementations modeled image formation (physiological optics) for planar or distant scenes. The ISET3d software described here extends that implementation, simulating image formation for three-dimensional scenes. The software system relies on a quantitative computer graphics program that ray traces the scene radiance through the physiological optics to the retinal irradiance. We describe and validate the implementation for several model eyes. Then, we use the software to quantify the impact of several physiological optics parameters on three-dimensional image formation. ISET3d is integrated with ISETBio, making it straightforward to convert the retinal irradiance into cone excitations. These methods help the user compute the predictions of optics models for a wide range of spatially-rich three-dimensional scenes. They can also be used to evaluate the impact of nearby visual occlusion, the information available to binocular vision, or the retinal images expected from near-field and augmented reality displays.

neuroscience↗

bioRxiv2018

The relative contribution of color and material in object selection

Information about color, material, texture and shape guide how we interact with objects. We developed a paradigm that quantifies how two object properties (color and material) combine in object selection. On each experimental trial, observers viewed three blob-shaped objects -- the target and two tests -- and selected the test that was more similar to the target. Across trials the target was fixed while the tests varied in color and material. We present a novel observer model that allows us to describe observers selection data in terms of (1) the underlying perceptual stimulus representation and (2) a color-material weight, which quantifies the relative importance of color vs. material in selection. We document large individual differences in the color-material weight. Furthermore, our analyses reveal limits on how precisely selection data simultaneously constrain perceptual representations and the color-material weight. These limits should guide future efforts towards understanding the multidimensional nature of object perception.

neuroscience↗

bioRxiv2018

Computational Luminance Constancy from Naturalistic Images

The human visual system supports stable percepts of object color even though the light that reflects from object surfaces varies significantly with the scene illumination. To understand the computations that support stable color perception, we study how estimating a target objects luminous reflectance factor (LRF; a measure of the light reflected from the object under a standard illuminant) depends on variation in key properties of naturalistic scenes. Specifically, we study how variation in target object reflectance, illumination spectra, and the reflectance of back-ground objects in a scene impact estimation of a target objects LRF. To do this, we applied supervised statistical learning methods to the simulated excitations of human cone photoreceptors, obtained from labeled naturalistic images. The naturalistic images were rendered with computer graphics. The illumination spectra of the light sources and the reflectance spectra of the surfaces in the scene were generated using statistical models of natural spectral variation. Optimally decoding target object LRF from the responses of a small learned set of task-specific linear receptive fields that operate on a contrast representation of the cone excitations yields estimates that are within 13% of the correct LRF. Our work provides a framework for evaluating how different sources of scene variability limit performance on luminance constancy.

neuroscience↗

bioRxiv2018

Computational-Observer Analysis of Illumination Discrimination

The spectral properties of the ambient illumination provide useful information about time of day and weather. We study the perceptual representation of illumination by analyzing measurements of how well people discriminate between illuminations across scene configurations. More specifically, we compare human performance to a computational-observer analysis that evaluates the information available in the isomerizations of the cones in a model human photoreceptor mosaic. Some patterns of human performance are predicted by the computational observer, other aspects are not. The analysis clarifies which aspects of performance require additional explanation in terms of the action of visual mechanisms beyond the isomerization of light by the cones.

neuroscience↗

bioRxiv2018

The perception of color and material in natural tasks

Perceived object color and material properties help us to select and interact with objects. Because there is no simple mapping between the pattern of an objects image on the retina and its physical reflectance, our perception of color and material are made more useful through sophisticated visual computations. A long-standing goal in vision science is to describe how these computations work, particularly as they act to stabilize perceived color and material against variation in scene factors extrinsic to object surface properties, such as the illumination. If we take seriously the notion that perceived color and material are useful because they help guide behavior in natural tasks, then we need experimental that measure and models that describe how they are used in such tasks. To this end, we have developed selection-based methods and accompanying perceptual models for studying perceived object color and material. This focused review highlights key aspects of our work. It includes a discussion of future directions and challenges, as well as an outline of a computational observer model that incorporates early, known, stages of visual processing and that clarifies how early vision shapes selection performance.

neuroscience↗

bioRxiv2018

Spatial summation in the human fovea: the effect of optical aberrations and fixational eye movements

Psychophysical inferences about the neural mechanisms supporting spatial vision can be undermined by uncertainties introduced by optical aberrations and fixational eye movements, particularly in fovea where the neuronal grain of the visual system is fine. We examined the effect of these pre-neural factors on photopic spatial summation in the human fovea using a custom adaptive optics scanning light ophthalmoscope that provided control over optical aberrations and retinal stimulus motion. Consistent with previous results, Riccos area of complete summation encompassed multiple photoreceptors when measured with ordinary amounts of ocular aberrations and retinal stimulus motion. When both factors were minimized experimentally, summation areas were essentially unchanged, suggesting that foveal spatial summation is limited by post-receptoral neural pooling. We compared our behavioral data to predictions generated with a physiologically-inspired front-end model of the visual system, and were able to capture the shape of the summation curves obtained with and without pre-retinal factors using a single post-receptoral summing filter of fixed spatial extent. Given our data and modeling, neurons in the magnocellular visual pathway, such as parasol ganglion cells, provide a candidate neural correlate of Riccos area in the central fovea.

neuroscience↗

bioRxiv2017

Simulation of visual perception and learning with a retinal prosthesis

The nature of artificial vision with a retinal prosthesis, and the degree to which the brain can adapt to the unnatural input from such a device, are poorly understood. Therefore, the development of current and future devices may be aided by theory and simulations that help to infer and understand what prosthesis patients see. A biologically-informed, extensible computational framework is presented here to predict visual perception and the potential effect of learning with a subretinal prosthesis. The framework relies on optimal linear reconstruction of the stimulus from retinal responses to infer the visual information available to the patient. A simulation of the physiological optics of the eye and light responses of the major retinal neurons was used to calculate the optimal linear transformation for reconstructing natural images from retinal activity. The result was then used to reconstruct the visual stimulus during the artificial activation expected from a subretinal prosthesis in a degenerated retina, as a proxy for inferred visual perception. Several simple observations reveal the potential utility of such a simulation framework. The inferred perception obtained with prosthesis activation was substantially degraded compared to the inferred perception obtained with normal retinal responses, as expected given the limited resolution and lack of cell type specificity of the prosthesis. Consistent with clinical findings and the importance of cell type specificity, reconstruction using only ON cells, and not OFF cells, was substantially more accurate. Finally, when reconstruction was re-optimized for prosthesis stimulation, simulating the greatest potential for learning by the patient, the accuracy of inferred perception was much closer to that of healthy vision. The reconstruction approach thus provides a more complete method for exploring the potential for treating blindness with retinal prostheses than has been available previously. It may also be useful for interpreting patient data in clinical trials, and for improving prosthesis design.

neuroscience↗