Face identity selectivity in the deep neural network and human brain

Abstract

A central challenge in face perception research is to understand how neurons encode various face identities. However, this challenge has not been met largely due to the lack of simultaneous access to the activity of the entire face processing neural network as well as the lack of a comprehensive multifaceted model that is able to characterize a large number of facial features. In this study, we address this challenge by conducting in silico experiments using a deep neural network (DNN) capable of face recognition with a diverse array of stimuli. We identified a subset of DNN neurons selective to face identities, and these identity-selective neurons demonstrated generalized discriminability to novel faces not involved in the training and in many different styles. Visualization of the network explained the response of the DNN neurons and manipulation of the network confirmed the importance of identity-selective neurons in face recognition. Importantly, using our human single-neuron recordings, we directly compared the response of artificial neurons with 490 real human neurons to the same stimuli and found that artificial neurons did share a similar representation of facial features as human neurons. We also observed a novel region-based feature coding mechanism in DNN neurons as in human neurons, which may explain how the DNN performs face recognition. Together, by directly linking between artificial and human neurons, our results shed light on how human neurons encode face identities.