Framed RSA: Representational comparisons that honor bothgeometry and population-mean response preferences

Abstract

Representational similarity analysis (RSA) characterizes the geometry of neural activity patterns elicited by different stimuli while discarding information about neural response preferences, regional population-mean activity, and the absolute location and orientation of the patterns in the multivariate response space. When evaluating alternative representational models, invariance to certain aspects of the neural code is desirable because systems might use superficially different encodings to implement the same computations. However, neural preferences and regional-mean activation are arguably physiologically and mechanistically important, and so we may want our models to predict them correctly. Here we introduce a novel analysis technique, framed RSA, which honors both geometry and population-mean preferences in evaluating model-predicted representations. To achieve this, we augment the set of patterns that define the geometry by two reference patterns: the all-zero point (origin) and an all-c (uniform constant) pattern in the multivariate response space, enabling RSA to incorporate information about the profile across stimuli of regional population-mean activations and about the global location and orientation of the ensemble of response patterns. We show that framed RSA improves model-selection accuracy when ground truth is known, considering brain-region identification (using human fMRI data from the Natural Scenes Dataset and macaque intracranial recording data from the Things Ventral Stream Spiking Dataset) and deep-neural-network-layer identification. By incorporating neural population preferences into model evaluation, framed RSA enables more mechanistically meaningful model comparisons and benefits from improved power for model-comparative inference.