No benefit of subsampling in ensemble coding

Abstract

Our visual system compensates for its limited information processing capacity by creating summary representations of stimulus ensembles. A longstanding debate concerns whether summary statistics are computed by integrating information across the entire ensemble or by sampling only a subset of its items. Subsampling is intuitively appealing because 1) integrating over a small subset presumably reduces processing resource expenditure and 2) observer models employing this strategy can effectively reproduce ensemble coding performance. Here, we systematically evaluate both aspects. We first show that the apparent success of subsampling models depends on a problematic assumption: that the sensory encoding noise of each item remains constant as set size increases. This assumption entails that total coding resources, quantified as Fisher information, grow unconstrained with set size. Under the realistic assumption that total coding resources are limited, such that the encoding precision for each item decreases with increasing set size, we demonstrate that an ideal observer integrating the entire ensemble achieves the same accuracy with a strictly smaller total resource budget for every set size. We further show that this resource-constrained full-integration model provides a more parsimonious, yet equally good, account of ensemble coding behavior across multiple existing datasets as well as newly collected data. Our findings call into question the necessity of the subsampling hypothesis, as restricting integration to a subset of ensemble items confers neither a theoretical nor an explanatory benefit over resource-rational full integration.