A Relational Code for Objects in Primate Frontal Cortex

Abstract

When we remember a scene, we recall both the locations of individual objects and their spatial arrangement. Yet, how neurons encode this arrangement alongside individual locations is unknown. In this work, we recorded large-scale neural activity from the frontal and supplementary eye fields of monkeys performing a two-object spatial working-memory task. Using a continuum of two object configurations, we tested whether neurons encoded individual object locations or the spatial relationship between them. Many neurons had well-defined position tuning when a single object was presented. With two objects, responses spanned a continuum from retinotopic to relational tuning. Surprisingly, most units were better explained by relational tuning than retinotopic tuning. Relational and retinotopic units encoded complementary dimensions of the stimulus, and their observed mixture supported near-optimal population decoding of object locations. Relational tuning was systematically aligned with single-object receptive fields, enabling a readout that produced repulsive biases resembling those observed in behavior. Standard multi-object models such as averaging and normalization failed to reproduce the observed continuum, whereas a model integrating receptive fields over potential saccades captured it. These findings show that the frontal oculomotor cortex represents not only where objects are, but also how they are arranged with respect to one another.