Vision-language representations link food perception to subjective valuation

Abstract

Subjective values of foods guide dietary decisions. Yet, how the brain computes such values from visual input remains unclear. Here, we show that visual food valuation relies on higher-level conceptual representations captured by a multimodal vision-language deep neural network model. Using food images and human ratings (N = 194 and 200), we found that a vision-language model predicted subjective value more accurately than vision-only models. Representational analyses demonstrated that, compared with the vision-only model, the vision-language model preferentially encoded higher-level food attributes, including perceived healthiness, while encoding less image-level visual information. Multivariate analyses of fMRI data (N = 31) showed that embedding features from both models corresponded to activity patterns along the visual pathways. Critically, vision-language features showed greater correspondence than vision-only features in the fusiform gyrus, a downstream region of the ventral visual pathway. These findings suggest that higher-level conceptual representations link visual food processing to subjective valuation.