Spatiotemporal Decoding of Explore-Exploit Decisions in the Human Brain

Abstract

Adaptive behavior requires flexibly shifting between exploiting familiar rewards and exploring novel opportunities. These explore-exploit decisions are implemented via a distributed brain network--anchored in frontopolar cortex (FPC) and ventromedial prefrontal cortex (vmPFC)--that computes the total value of a given choice by weighting the immediate value of familiar options against the latent future value of exploration. Capturing the precise temporal dynamics of these neural computations across spatially distributed cortical networks is critical to resolving how the brain manages this tradeoff. Here, we combined magnetoencephalography (MEG) with partially observable Markov decision process (POMDP) modeling during a reinforcement learning task. By decoding POMDP-derived choice policies in MEG source space, we mapped the precise spatiotemporal emergence of explore-exploit decisions across the cortex. We demonstrate that explore-exploit policy implementation unfolds via a hierarchical functional dissociation across the rostral prefrontal cortex. During decision formation, the lateral FPC initiates the strategic shift toward exploration hundreds of milliseconds before choice execution. Conversely, vmPFC and orbitofrontal cortex (OFC) do not exhibit early significant divergence from the baseline exploitative trace on exploration trials, displaying only a delayed, transient peak prior to choice execution. Following feedback, the vmPFC and OFC transition to a sustained representation of the choices empirical value to update reward expectancies and guide future actions. Mapping reinforcement learning variables onto millisecond-resolved neural data reveals how the human brain resolves the explore-exploit dilemma: through early strategic initiation of exploration in the FPC, a delayed exploratory shift in vmPFC and OFC, and sustained outcome evaluation to optimize future actions.