Biology subjects

Brainard, D. H.

Publications and source records attributed to Brainard, D. H..

At least 19 recordsLinked to original sources

bioRxiv2026

The interplay between detection and localization in human vision

Detection and localization are fundamental but distinct tasks of sensory systems: detecting a signal requires accumulating evidence for its presence, whereas localizing it requires extracting spatial information. How active sampling should be organized when both tasks must be performed simultaneously remains unclear. Here, using an analytically tractable model of early visual processing and Bayesian ideal-observer inference, we establish a direct relation between the two tasks: localizing a stimulus is equivalent to detecting its spatial gradient. Consequently, detection and localization favor fundamentally different sampling dynamics when the stimulus size exceeds the effective blur scale of the visual system. Applied to fixational eye movements (FEMs), which continually translate stationary visual stimuli across the adapting retina, this distinction yields two competing optimal movement scales. Localization is optimized when the eye moves approximately one retinal blur length during the adaptation time, whereas detection is optimized by motion on the scale of the stimulus size. The scale of physiological FEMs lies near the predicted localization optimum, while stimulus detection remains comparatively robust to variations in eye motion. Our model recovers established laws of temporal and spatial summation while predicting additional scaling regimes arising from FEMs. We propose an experimental paradigm that tests these predictions by measuring localization at matched detectability, eliminating the unknown internal-noise amplitude. Our results reveal a fundamental trade-off between acquiring evidence for the presence and position of spatially extended signals and suggest that human fixational eye movements preferentially support spatial localization.

neuroscience↗

bioRxiv2026

Model-optimized stimulus distortions for adaptive estimation of individual sensory representations

Representations of the same physical stimulus vary between individuals. Characterizing individual differences has practical implications, but is challenging because these representations are not directly observable. Given a model of how representations vary within a population, we propose a Bayesian adaptive procedure for estimating an individual observers representation from a series of targeted perceptual discrimination judgments. A key component of our approach is using Fisher information to identify stimulus distortions that efficiently differentiate observers in the population. As a proof of concept, we focus on individual differences in color perception and simulate observers with cone fundamentals drawn from an individual colorimetric observer model. We demonstrate that our approach can recover key aspects of a sampled observers cone fundamentals using simulated three-alternative forced-choice oddity judgments with approximately 500 trials, corresponding to an experimental duration of approximately one hour. Our Bayesian adaptive framework provides a promising and generalizable approach to efficiently link behavioral measurements to individual differences in sensory representations.

neuroscience↗

bioRxiv2026

Transformation-tolerant object recognition in tree shrews despite lacking a fovea

Object recognition depends on the ability to extract stable representations across changes in how they are viewed, yet it remains unclear how this capacity depends on visual acuity and cortical hierarchy. We combined behavioral testing and computational modeling to determine whether tree shrews, close relatives of primates with lower spatial acuity, can perform transformation-tolerant object recognition. Front-end modeling incorporating species-specific optics and photoreceptor sampling showed that, when scaled for acuity, tree shrew retinal filtering preserves the similarity structure of natural image categories relevant for object recognition. Behaviorally, tree shrews reliably discriminated complex objects across variations in position, scale, and viewpoint, including when embedded within natural scenes, and generalized to novel exemplars. Their recognition behavior was best explained by visual features emphasizing differences in global shape and size between objects and by representations from intermediate and deep layers of hierarchical neural network models. These results demonstrate that visual processing supporting object-level generalization can arise within visual systems lacking high-acuity front-end optics and establish the tree shrew as a key model for understanding the computational and evolutionary origins of high-level vision.

animal behavior and cognition↗

bioRxiv2025

A deep convolutional neural network trained for lightness constancy is susceptible to lightness illusions

Human viewers are able to perform tasks that depend on accurate estimates of surface reflectance, even across large changes in illumination and context. This is a remarkable ability, and successful image-computable models of how the visual system achieves this have remained elusive. Recently, deep convolutional neural networks (CNNs) have been developed that are adept at estimating surface reflectance. Here we evaluated one such network as a starting point for a new model of human lightness perception by testing whether it was susceptible to a range of classic lightness illusions. We implemented a CNN and trained it via supervised learning to estimate surface reflectance at each pixel in grayscale, rendered images of geometric objects. We examined the networks output on several illusions, including the argyle, Koffka, snake, simultaneous contrast, Whites, and checkerboard illusions, as well as control figures. We included variants where low-luminance regions important to the illusions were generated either by low reflectance or by cast shadows. For comparison, we carried out a lightness matching experiment with human observers using the same stimuli, and also examined the outputs of three classic lightness and brightness models. The CNN largely removed lighting effects such as shading and shadows, and produced good reflectance estimates on a test set. It also qualitatively predicted the illusions perceived by humans in most cases, the exceptions being Whites and checkerboard illusions. The CNN outperformed classical models, both at estimating reflectance and at tracking human lightness matches. These findings support a normative view of lightness perception and highlight the promise of deep learning models in this area.

neuroscience↗

bioRxiv2025

An image-computable, spatio-chromatic RF model of the midget RGC mosaic across the retina

Accurate image-computable models of retinal ganglion cell (RGC) mosaics across the retina do not currently exist. Here, we deploy a novel computational frame-work which synthesizes mosaics of linear spatio-chromatic receptive fields (RFs) of ON midget RGCs (mRGCs) by integrating published anatomical, physiological, and optical quality measurements. We use the synthesized mRGC mosaics to simulate both in vivo and in vitro physiological experiments and demonstrate the models consistency with published data. The model enables computation of how visual performance is shaped by the representation of visual information provided by the linear spatiochromatic processing stage of midget RGCs. The developed computational framework carefully accounts for the effect of physiological optics on mRGC responses, enables comparison of in vivo and in vitro data, and allows exploration of how different assumptions about RF organization, such as selectivity for the type of cones pooled by the RF center mechanism, affect physiological responses and psychophysical performance. The open-source and freely available implementation provides a platform for understanding how the linear spatiochromatic receptive field representation of the mRGCs shapes visual performance, as well as a foundation for future work that incorporates response nonlinearities, temporal filtering, and extends to additional RGC mosaics.

neuroscience↗

bioRxiv2025

Information, Movement and Adaptation in Human Vision

Our eyes are never still. Even when fixating, they exhibit small, jittery motions. While it has long been argued that these fixational eye movements (FEMs) aid the acquisition of visual information, a complete theoretical description of their impact on the information available in the early visual system has been lacking. Here we build FEMs into a minimal theoretical model of the early visual response, including the critical process of adaptation (a fading response to a fixed image). We establish the effect of FEMs on the mutual information between a visual stimulus and this response. Our approach identifies the key dimensionless parameters that characterise the effect of fixational eye movements, and reveals the regimes in which this effect can be beneficial, detrimental or negligible. Taking parameter values appropriate for human vision, we show that the spatio-temporal couplings induced by fixational eye movements can explain the qualitative features of the human contrast sensitivity function as well as classic experiments on temporal integration. To our knowledge, the consideration of fixational eye movements in this latter context is novel, and our results suggest the need for future experiments to determine the mechanisms by which spatial and temporal responses are coupled in the human visual system.

neuroscience↗

bioRxiv2025

Comprehensive characterization of human color discrimination thresholds

Color discrimination thresholds--the smallest detectable color differences--provide a benchmark for models of color vision, enable quantitative evaluation of eye diseases, and inform the design of display technologies. Despite their importance, a comprehensive characterization of these thresholds has long been considered intractable due to the psychophysical curse of dimensionality. Here, we address this challenge using a novel semi-parametric Wishart Process Psychophysical Model (WPPM), which leverages the feature that the internal noise limiting color discrimination varies smoothly across stimulus space. The model was fit to data collected with a non-parametric adaptive trial-placement procedure, enabling efficient stimulus selection. Together, through the combination of adaptive trial placement and post hoc WPPM fitting, we achieved a comprehensive characterization of color discrimination in the isoluminant plane with only [~]6,000 trials per participant (N = 8). Once fit, the WPPM allows readouts of discrimination performance for any stimulus pair. We validated these readouts against 25 probe psychometric functions, measured with an additional 6,000 trials per participant held out from model fitting. In conclusion, our study provides a foundational dataset for color vision, and our approach generalizes beyond color to any domain in which the internal noise limiting performance varies smoothly across stimulus space, offering a powerful and efficient method for comprehensively characterizing various perceptual discrimination thresholds.

neuroscience↗

bioRxiv2024

Temporal dynamics of human color processing measured using a continuous tracking task

We characterized the temporal dynamics of color processing using a continuous tracking paradigm by estimating temporal impulse response functions associated with tracking chromatic Gabor patches. We measured how the lag of these functions changes as a function of chromatic direction and contrast for stimuli in the LS cone contrast plane. In the same set of subjects, we also measured detection thresholds for stimuli with matched spatial, temporal, and chromatic properties. We created a model of tracking and detection performance to test if a common representation of chromatic contrast accounts for both measures. The model summarizes the effect of chromatic contrast over different chromatic directions through elliptical isoresponse contours, the shapes of which are contrast independent. The fitted elliptical isoresponse contours have essentially the same orientation in the detection and tracking tasks. For the tracking task, however, there is a striking reduction in sensitivity to signals originating in the S cones. The results are consistent with common chromatic mechanisms mediating performance on the two tasks, but with task-dependent relative weighting of signals from L and S cones.

neuroscience↗

bioRxiv2024

Orthogonal neural representations support perceptual judgements of natural stimuli

In natural behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on simple backgrounds. Natural viewing, however, carries a set of challenges that are inaccessible using artificial stimuli, including neural responses to background objects that are task-irrelevant. An emerging body of evidence suggests that the visual abilities of humans and animals can be modeled through the linear decoding of task-relevant information from visual cortex. This idea suggests the hypothesis that irrelevant features of a natural scene should impair performance on a visual task only if their neural representations intrude on the linear readout of the task relevant feature, as would occur if the representations of task-relevant and irrelevant features are not orthogonal in the underlying neural population. We tested this hypothesis using human psychophysics and monkey neurophysiology, in response to parametrically variable naturalistic stimuli. We demonstrate that 1) the neural representation of one feature (the position of a central object) in visual area V4 is orthogonal to those of several background features, 2) the ability of human observers to precisely judge object position was largely unaffected by task-irrelevant variation in those background features, and 3) many features of the object and the background are orthogonally represented by V4 neural responses. Our observations are consistent with the hypothesis that orthogonal neural representations can support stable perception of objects and features despite the tremendous richness of natural visual scenes. Significance StatementWe studied how the structure of the mid-level neural representation of multiple visual features supports robust perceptual decisions. We combined array recording with parametrically controlled naturalistic images to demonstrate that the representation of a central objects position in monkey visual area V4 is orthogonal to that of several background features. In addition, we used human psychophysics with the same stimulus set to show that observers ability to judge a central objects position is largely unaffected by variation in the same background features. This result supports the hypothesis that orthogonal neural representations can enable stable and robust perception in naturalistic visual environments and advances our understanding of how visual processing operates in the real world.

neuroscience↗

bioRxiv2024

Deriving the cone fundamentals: a subspace intersection method

Two ideas, proposed by Thomas Young and James Clerk Maxwell, form the foundations of color science: (1) Three types of retinal receptors encode light under daytime conditions, and (2) color matching experiments establish the critical spectral properties of this encoding. Experimental quantification of these ideas are used in international color standards. But, for many years the field did not reach consensus on the spectral properties of the biological substrate of color matching: the sensitivity of the in situ cones (cone fundamentals). By combining auxiliary data (thresholds, inert pigment analyses), complex calculations, and color matching from genetically analyzed dichromats, the human cone fundamentals have now been standardized. Here we describe a new computational method to estimate the cone fundamentals using only color matching from dichromatic observers. We show that it is not necessary to include data from trichromatic observers in the analysis or to know the primary lights used in the matching experiments. Remarkably, it is even possible to estimate the fundamentals by combining data from experiments using different, unknown primaries. We then suggest how the new method may be applied to color management in modern image systems.

neuroscience↗

bioRxiv2022

In vivo physiology of foveal retinal ganglion cells in Macaca fascicularis

The primate fovea is specialized for high acuity chromatic vision, with the highest density of cone photoreceptors and a disproportionately large representation in visual cortex. The unique visual properties conferred by the fovea are conveyed to the brain by retinal ganglion cells, the somas of which lie at the margin of the foveal pit. Microelectrode recordings of these centermost retinal ganglion cells have been challenging due to the fragility of the fovea in the excised retina. Here we overcome this challenge by combining high resolution fluorescence adaptive optics ophthalmoscopy with calcium imaging to optically record functional responses of foveal retinal ganglion cells in the living eye. We use this approach to study the chromatic responses and spatial transfer functions of retinal ganglion cells using spatially uniform fields modulated in different directions in color space and monochromatic drifting gratings. We recorded from over 350 cells across three Macaca fascicularis primates over a time period of weeks to months. We find that the majority of the L vs. M cone opponent cells serving the most central foveolar cones have spatial transfer functions that peak at high spatial frequencies (20-40 c/deg), reflecting strong surround inhibition that sacrifices sensitivity at low spatial frequencies but preserves the transmission of fine detail in the retinal image. In addition, we fit to the drifting grating data a detailed model of how ganglion cell responses draw on the cone mosaic to derive receptive field properties of L vs. M cone opponent cells at the very center of the foveola. The fits are consistent with the hypothesis that foveal midget ganglion cells are specialized to preserve information at the resolution of the cone mosaic. By characterizing the functional properties of retinal ganglion cells in vivo through adaptive optics, we characterize the response characteristics of these cells in situ.

neuroscience↗

bioRxiv2021

Melanopic stimulation does not affect psychophysical threshold sensitivity for luminance flicker

In addition to the cone photoreceptors the retina contains intrinsically photosensitive retinal ganglion cells (ipRGCs). These cells express the photopigment melanopsin and are known to be involved in reflexive visual functions such as pupil response and photo-entrainment of the circadian rhythm. It is possible that the ipRGCs contribute to conscious visual perception, either by providing an independent signal to the geniculo-striate pathway, or by interacting with and thus modifying signals arising from "classical" retinal ganglion cells that combine and contrast cone input. Here, we tested for the existence of an interaction by asking if a 350% change in melanopsin stimulation alters psychophysical sensitivity for the detection of luminance flicker. In Experiment 1, we tested for a change in the threshold for detecting luminance flicker in three participants after they adapted to backgrounds with different degrees of tonic melanopsin stimulation. In Experiments 2 and 3, this test was repeated, but now for luminance flicker presented on a transient pedestal of melanopsin stimulation. Across the three experiments, no effect of melanopsin stimulation upon threshold flicker sensitivity was found. Our results suggest that even large changes in melanopsin stimulation do not affect near-threshold, cone-mediated visual perception.

neuroscience↗

bioRxiv2021

Equivalent noise characterization of human lightness constancy

A goal of visual perception is to provide stable representations of task-relevant scene properties (e.g. object reflectance) despite variation in task-irrelevant scene properties (e.g. illumination, reflectance of other nearby objects). To study such stability in the context of the perceptual representation of lightness, we introduce a threshold-based psychophysical paradigm. We measure how thresholds for discriminating the achromatic reflectance of a target object (task-relevant property) in rendered naturalistic scenes are impacted by variation in the reflectance functions of background objects (task-irrelevant property), using a two-alternative forced-choice paradigm in which the reflectance of the background objects is randomized across the two intervals of each trial. We control the amount of background reflectance variation by manipulating a statistical model of naturally-occurring surface reflectances. For low background object reflectance variation, discrimination thresholds were nearly constant, indicating that observers internal noise determines threshold in this regime. As background object reflectance variation increases, its effects start to dominate performance. A model based on signal detection theory allows us to express the effects of task-irrelevant variation in terms of the equivalent noise, that is relative to the intrinsic precision of the task-relevant perceptual representation. The results indicate that although naturally-occurring background object reflectance variation does intrude on the perceptual representation of target object lightness, the effect is modest - within a factor of two of the equivalent noise level set by internal noise.

neuroscience↗

bioRxiv2021

An Image Reconstruction Framework for Characterizing Early Vision

We developed an image-computable observer model of the initial visual encoding that operates on natural image input, based on the framework of Bayesian image reconstruction from the excitations of the retinal cone mosaic. Our model extends previous work on ideal observer analysis and evaluation of performance beyond psychophysical discrimination, takes into account the statistical regularities of the visual environment, and provides a unifying framework for answering a wide range of questions regarding the visual front end. Using the error in the reconstructions as a metric, we analyzed variations of the number of different photoreceptor types on human retina as an optimal design problem. In addition, the reconstructions allow both visualization and quantification of information loss due to physiological optics and cone mosaic sampling, and how these vary with eccentricity. Furthermore, in simulations of color deficiencies and interferometric experiments, we found that the reconstructed images provide a reasonable proxy for modeling subjects percepts. Lastly, we used the reconstruction-based observer for the analysis of psychophysical threshold, and found notable interactions between spatial frequency and chromatic direction in the resulting spatial contrast sensitivity function. Our method is widely applicable to experiments and applications in which the initial visual encoding plays an important role.

neuroscience↗

bioRxiv2020

Enhanced reflexive eye closure is mediated by ipRGC signals and is specific to migraine with visual aura

ObjectiveTo assess the contribution of the melanopsin-containing, intrinsically photosensitive retinal ganglion cells (ipRGCs) and the cones to reflexive eye closure as an implicit measure of interictal photophobia in migraine. MethodsWe studied twenty participants in each of three groups: headache-free (HAf) controls, migraine without aura (MwoA), and migraine with visual aura (MwA). Participants viewed spectral pulses that selectively targeted melanopsin, the cones, or their combination while we recorded orbicularis oculi electromyography (OO-EMG) and blinking rate. ResultsTime course analysis of OO-EMG demonstrated that reflexive eye closure was tightly coupled to the spectral pulses. Compared to both the MwoA and HAf control groups, the MwA group had enhanced OO-EMG and blinking activity in response to melanopsin and cone stimulation in combination and in isolation. This response scaled with the contrast of the stimulus. ConclusionsOur findings suggest that ipRGC signals, whether elicited by melanopsin stimulation or from presumed extrinsic cone input, provide the afferent input for light-induced reflexive eye closure in a photophobic state. Participants with migraine and visual aura had a distinctly different response to visual stimulation as compared to the other two groups. This is in contrast to prior findings in this same cohort in whom higher explicit ratings of visual discomfort were found for both MwA and MwoA as compared to controls. Such a dissociation suggests distinct pathophysiology in forms of migraine, interacting with separate neural pathways by which ipRGC signals elicit implicit and explicit signs of visual discomfort.

neuroscience↗

bioRxiv2020

A Quadratic Model Captures the Human V1 Response to Variations in Chromatic Direction and Contrast

An important goal for vision science is to develop quantitative models for the representation of visual signals at post-receptoral sites. To this end, we develop the quadratic color model (QCM) and examine its ability to account for the BOLD fMRI response in human V1 to spatially-uniform temporal chromatic modulations, systematically varying in their chromatic directions and contrasts. We find that the QCM explains the same, cross-validated variance as a conventional GLM, with far fewer free parameters. The QCM generalizes to allow prediction of V1 responses to a large range of modulations. We replicated the results for each subject and find good agreement across both replications and subjects. We find that within the LM cone contrast plane, V1 is most sensitive to L-M contrast modulations and least sensitive to L+M contrast modulations. Within V1, we observe little to no change in chromatic sensitivity as a function of eccentricity.

neuroscience↗

bioRxiv2020

Selective amplification of ipRGC signals accounts for interictal photophobia in migraine

Second only to headache, photophobia is the most debilitating symptom reported by people with migraine. While the melanopsin-containing, intrinsically photosensitive retinal ganglion cells (ipRGCs) are thought to play a role, how cone and melanopsin signals are integrated in this pathway to produce visual discomfort is poorly understood. We studied 60 people: 20 without headache and 20 each with interictal photophobia from migraine with or without aura. Participants viewed pulses of spectral change that selectively targeted melanopsin, the cones, or both, and rated the degree of visual discomfort produced by these stimuli while we also recorded pupil responses. We examined the data within a model that describes how cone and melanopsin signals are weighted and combined at the level of the retina, and how this combined signal is transformed into a rating of discomfort or pupil response. Our results indicate that people with migraine do not differ from headache-free controls in the manner in which melanopsin and cone signals are combined. Instead, people with migraine demonstrate an amplification of integrated ipRGC signals for discomfort. This effect of migraine is selective for ratings of visual discomfort, in that an amplification of pupil responses was not seen in the migraine group, nor were group differences found in surveys of other behaviors putatively linked to ipRGC function (chronotype, seasonal sensitivity, presence of a photic sneeze reflex). By revealing a dissociation in the amplification of discomfort versus pupil response, our findings suggest a post-retinal alteration in processing of ipRGC signals for photophobia in migraine. SignificanceThe melanopsin-containing, intrinsically photosensitive retinal ganglion cells (ipRGCs) may contribute to photophobia in migraine. We measured visual discomfort and pupil responses to cone and melanopsin stimulation--the photoreceptor inputs to the ipRGCs--in people with and without migraine. We find that people with migraine do not differ from those without headaches in how cone and melanopsin signals are weighted and combined to produce visual discomfort. Instead, migraine is associated with an amplification of ipRGC signals for discomfort. This effect of migraine upon ipRGC signals is limited to photophobia, as we did not find an enhancement of pupil responses or a change in other behaviors linked to ipRGC function. Our findings suggest a post-retinal amplification of ipRGC signals for photophobia in migraine.

neuroscience↗

bioRxiv2019

A computational observer model of spatial contrast sensitivity: Effects of photocurrent encoding, fixational eye movements and inference engine

We have recently shown that using the information carried by the mosaic of cone excitations of a stationary retina, the relative spatial contrast sensitivity function (CSF) of a computational observer has the same shape as a typical human subject. Absolute human sensitivity, however, is lower than the computational observer by a factor of 5 to 10. Here we model how additional known features of early vision affect spatial contrast sensitivity: fixational eye movements and the conversion of cone photopigment excitations to cone photocurrent responses. For a computational observer that uses a linear classifier applied to the responses of a stimulus-matched linear filter, fixational eye movements substantially change the shape of the spatial CSF, primarily by reducing sensitivity at spatial frequencies above 10 c/deg. For a computational observer that uses a translation-invariant calculation, in which decisions are based on the squared response of a quadrature-pair of linear filters, the CSF shape is little changed by eye movements, but there is a two-fold reduction in sensitivity. The noise and response dynamics of conversion of cone excitations into photocurrent introduce an additional two-fold sensitivity decrease. Hence, the combined effects of fixational eye movements and phototransduction bring the absolute sensitivity of the translation-invariant computational observer CSF to within a factor of 1 to 2 of the human CSF. We note that the human CSF depends on processing of the initial representation by many thalamic and cortical neurons, which are individually quite noisy. Our computational modeling suggests that the net effect of this noise on contrast-detection performance, when considered at the neural population level and behavioral level, is quite small: the inference mechanisms that determine the CSF, presumably in cortex, make efficient use of the information available from the cone photocurrents of the fixating eye.

neuroscience↗