Sharpened visual memory representations are reflected in inferotemporal cortex

Abstract

Humans and other primates can robustly report whether theyve seen specific images before, even when those images are extremely similar to ones theyve previously seen. Multiple lines of evidence suggest that pattern separation computations in the hippocampus (HC) contribute to this behavior by shaping the fidelity of visual memory. However, unclear is whether HC uniquely determines memory fidelity or whether computations in other brain areas also contribute. To investigate, we recorded neural signals from inferotemporal cortex (ITC) and HC of two rhesus monkeys as they performed a memory task in which they judged whether images were novel or exactly repeated in the presence of visually similar lure images with a range of visual similarities. We found behavioral evidence for sharpening, reflected as memory performance that was nonlinearly transformed relative to a benchmark defined by visual representations in ITC. As expected, we found that behavioral sharpening aligned with visual memory representations in HC. Surprisingly, and unaccounted for by HC pattern separation proposals, we also found neural correlates of behavioral sharpening reflected in ITC. These results, coupled with further analysis of the data, suggest that ITC contributes to shaping the fidelity of visual memory in the transformation from visual processing to memory storage and signaling. SignificanceVisual recognition memories are stored with remarkable visual fidelity, allowing humans and other primates to distinguish images they have encountered from visually similar images they have not. This fidelity has long been attributed to computations in the hippocampus that sharpen visual representations before memory storage ("pattern separation"). Unclear is how this proposal aligns with other evidence that visual memories are stored within high-level visual cortex itself, before signals reach the hippocampus. Here we demonstrate that, like the hippocampus, inferotemporal cortex also reflects sharpened visual memory representations, suggesting that visual cortex contributes to shaping the visual fidelity of visual memory.