Differentiating neural mechanisms in response to emotional expressions on real and virtual faces

Abstract

As avatars become more commonplace, understanding how the brain processes emotional expressions in virtual faces is critical. We compared behavioral and neural responses to real and virtual faces expressing seven emotions (anger, disgust, fear, joy, sadness, surprise, neutral). In Experiment 1 (n=61), participants rated the similarity between paired faces. Expressions conveying the same emotion were rated as highly similar across face types, whereas mismatched emotions yielded substantially lower similarity ratings, indicating perceived emotional meaning was preserved despite differences in face realism. In Experiment 2 (n=91), functional near-infrared spectroscopy was used to measure brain activity while participants viewed the same stimuli. General-linear-model analyses revealed greater activation limited to visual areas for 1) virtual faces and 2) surprise and neutral expressions. Functional connectivity analyses, however, revealed network level differences between face type and emotion across the brain. Real faces elicited stronger connectivity patterns across frontal, central-temporal, and parietal regions, whereas high-arousal emotions (fear, anger, and joy) were associated with broader network engagement than other expressions. Our results suggest face-type processing occur in early visual areas, and despite perceptual similarity, different emotions on real and virtual faces are associated with distinct patterns of network level connectivity across the brain.