Using deep convolutional neural networks to test why human face recognition works the way it does

Abstract

Human face recognition is highly accurate, and exhibits a number of distinctive and well documented behavioral "signatures" such as the use of a characteristic representational space, the disproportionate performance cost when stimuli are presented upside down, and the drop in accuracy for faces from races the participant is less familiar with. These and other phenomena have long been taken as evidence that face recognition is "special". But why does human face perception exhibit these properties in the first place? Here we use deep convolutional neural networks (CNNs) to test the hypothesis that all of these signatures of human face perception result from optimization for the task of face recognition. Indeed, as predicted by this hypothesis, these phenomena are all found in CNNs trained on face recognition, but not in CNNs trained on object recognition, even when additionally trained to detect faces while matching the amount of face experience. To test whether these signatures are in principle specific to faces, we optimized a CNN on car discrimination and tested it on upright and inverted car images. As for face perception, the car-trained network showed a drop in performance for inverted versus upright cars. Similarly, CNNs trained only on inverted faces produce an inverted inversion effect. These findings show that the behavioral signatures of human face perception reflect and are well explained as the result of optimization for the task of face recognition, and that the nature of the computations underlying this task may not be so "special" after all. Significance StatementFor decades, cognitive scientists have collected and characterized behavioral signatures of face recognition. Here we move beyond the mere curation of behavioral phenomena to asking why the human face system works the way it does. We find that many classic signatures of human face perception emerge spontaneously in CNNs trained on face discrimination, but not in CNNs trained on object classification (or on both object classification and face detection), suggesting that these long-documented properties of the human face perception system reflect optimizations for face recognition per se, not by-products of a generic visual categorization system. This work further illustrates how CNN models can be synergistically linked to classic behavioral findings in vision research, thereby providing psychological insights into human perception.