Application of a unifying reward-prediction error (RPE)-based framework to explain underlying dynamic dopaminergic activity in timing tasks

Abstract

Dopaminergic neurons (DANs) exhibit complex dynamics across a variety of behavioral contexts, often in ways that seem task-specific and even incompatible with results across different paradigms. Dopaminergic signaling during timing tasks has been a prime example. In behavioral timing, dopaminergic dynamics predict the initiation of self-timed movement via a seconds-long ramp up of activity prior to movement onset, similar to ramping seen in visuospatial reward approach and multi-step, goal-directed behaviors. By contrast, in perceptual timing, DANs exhibit more complex dynamics whose direction of modulation seems to be the opposite of that observed in behavioral timing. Mikhael et al. (2022) recently proposed a formal model in which dopaminergic dynamics encode reward expectation in the form of an "ongoing" reward-prediction error (RPE) that arises from resolving uncertainty of ones position in the value landscape (i.e., ones spatial-temporal distance to reward delivery/omission). Here, we show that application of this framework recapitulates and reconciles the seemingly contradictory dopaminergic dynamics observed in behavioral vs perceptual timing. These results suggest a common neural mechanism that broadly underlies timing behavior: trial-by-trial variation in the rate of the internal "pacemaker," manifested in DAN signals that reflect stretching or compression of the derivative of the subjective value function relative to veridical time. In this view, faster pacemaking is associated with relatively high amplitude dopaminergic signaling, whereas slower pacemaking is associated with relatively low levels of dopaminergic signaling, consistent with findings from pharmacological and lesion studies.