Murray, R. F. | bioRxiv Science

bioRxiv2025

A deep convolutional neural network trained for lightness constancy is susceptible to lightness illusions

Human viewers are able to perform tasks that depend on accurate estimates of surface reflectance, even across large changes in illumination and context. This is a remarkable ability, and successful image-computable models of how the visual system achieves this have remained elusive. Recently, deep convolutional neural networks (CNNs) have been developed that are adept at estimating surface reflectance. Here we evaluated one such network as a starting point for a new model of human lightness perception by testing whether it was susceptible to a range of classic lightness illusions. We implemented a CNN and trained it via supervised learning to estimate surface reflectance at each pixel in grayscale, rendered images of geometric objects. We examined the networks output on several illusions, including the argyle, Koffka, snake, simultaneous contrast, Whites, and checkerboard illusions, as well as control figures. We included variants where low-luminance regions important to the illusions were generated either by low reflectance or by cast shadows. For comparison, we carried out a lightness matching experiment with human observers using the same stimuli, and also examined the outputs of three classic lightness and brightness models. The CNN largely removed lighting effects such as shading and shadows, and produced good reflectance estimates on a test set. It also qualitatively predicted the illusions perceived by humans in most cases, the exceptions being Whites and checkerboard illusions. The CNN outperformed classical models, both at estimating reflectance and at tracking human lightness matches. These findings support a normative view of lightness perception and highlight the promise of deep learning models in this area.

neuroscience↗

bioRxiv2025

Deep neural networks trained for estimating albedo and illumination achieve lightness constancy differently than human observers.

Lightness constancy, the ability to create perceptual representations that are strongly correlated with surface albedo despite variations in lighting and context, is a challenging computational problem. Indeed, it has proven difficult to develop image-computable models of how human vision achieves a substantial degree of lightness constancy in complex scenes. Recently, convolutional neural networks (CNNs) have been developed that are proficient at estimating albedo, but little is known about how they achieve this, or whether they are good models of human vision. We examined this question by training a CNN to estimate albedo and illumination in a computer-rendered virtual world, and evaluating both the CNN and human observers in a lightness matching task. In several conditions, we eliminated cues potentially supporting lightness constancy: local contrast, shading, shadows, and all contextual cues. We found that the network achieved a high degree of lightness constancy, outperforming three classic models, and substantially outperforming human observers as well. However, we also found that eliminating cues affected the CNN and humans very differently. Humans had much worse constancy when local contrast cues were made uninformative, but were minimally affected by elimination of shading or shadows. The CNN was unaffected by local contrast, but relied on shading and shadows. These results suggest that the CNN followed an effective strategy of integrating global image cues, whereas humans used a more local strategy. In a follow-up experiment, we found that the CNN could learn to exploit noise artifacts that were correlated with illuminance in ray-traced scenes, whereas humans did not. We conclude that CNNs can learn an effective, global strategy of estimating lightness, which is closer to an optimal strategy for the ensemble of scenes we studied than the computation used by human vision.

neuroscience↗