Is categorization in visual working memory a way to reduce mental effort? A pupillometry study

Abstract

Recent studies on visual working memory (VWM) have shown that visual information can be stored in VWM as continuous (e.g., a specific shade of red) as well as categorical representations (e.g., the general category red). It has been widely assumed, yet never directly tested, that continuous representations require more VWM mental effort than categorical representations; given limited VWM capacity, this would mean that fewer continuous, as compared to categorical, representations can be maintained simultaneously. We tested this assumption by measuring pupil size, as a proxy for mental effort, in a delayed estimation task. Participants memorized one to four ambiguous (boundaries between adjacent color categories) or prototypical colors to encourage continuous or categorical representations, respectively; after a delay, a probe indicated the location of the to-be-reported color. We found that, for set size 1, pupil size was larger while maintaining ambiguous as compared to prototypical colors, but without any difference in memory precision; this suggests that participants relied on an effortful continuous representation to maintain a single ambiguous color, thus resulting in pupil dilation while preserving precision. In contrast, for set size 2 and higher, pupil size was equally large while maintaining ambiguous and prototypical colors, but memory precision was now substantially reduced for ambiguous colors; this suggests that participants now also relied on categorical representations for ambiguous colors (which are by definition a poor fit to any category), thus reducing memory precision but not resulting in pupil dilation. Taken together, our results suggest that continuous representations are more effortful than categorical representations, and that very few continuous representations (perhaps only one) can be maintained simultaneously. Significance statementVisual working memory (VWM) can store visual information in two more-or-less distinct formats: continuous and categorical representations. It is widely assumed that VWM capacity for continuous representations is more limited than for categorical representations, yet this assumption has never been explicitly tested. Here we address this crucial question of whether continuous representations cost more resources, and as a consequence limits VWM capacity, as compared to categorical representations. To do so, we measured pupil size (as a proxy for mental effort) and memory performance in a delayed estimation task. Crucially, our results suggest that continuous representations are more effortful than categorical representations, and that only one continuous representation can be maintained in VWM at one time.