Rouhani, N. | bioRxiv Science

bioRxiv2019

Joy division: Reward prediction errors create event boundaries in memory

We remember when things change. Particularly salient are experiences where there is a change in rewards, eliciting reward prediction errors (RPEs). This feature of memory may be useful because it can help us find greater rewards and avoid lesser ones in the future. How do RPEs influence our memory of those experiences? One idea is that this signal directly enhances the encoding of memory. Another, not mutually exclusive, idea is that the RPE signals a deeper change in the environment, and leads to the mnemonic separation of subsequent experiences from what came before, thereby creating a new latent context and a more separate memory trace. We tested this in four experiments in which participants learned to predict rewards associated with a series of images within visually-distinct \"rooms.\" High magnitude RPEs indicated a change in the underlying distribution of rewards. To test whether these large RPEs created a new latent context, we first assessed recognition priming for sequential pairs that contained or did not contain a high-RPE event, as well as out-of-sequence pairs (Exp. 1: n=27 & Exp. 2: n=83). We found evidence of recognition priming for both sequential pair types, including the pair with the high-RPE event, indicating that the high-RPE event is bound to its predecessor in memory. Given that high-RPE events are themselves preferentially remembered (Rouhani et al, 2018), we next tested recognition priming for pairs that had one item in between them (i.e. the pairs were either across a high-RPE event or not), where none of the tested items were high-RPE items (Exp. 3: n=85). Here, sequential pairs across a high-RPE no longer showed recognition priming whereas pairs within the same latent reward state did, providing initial evidence for an RPE-modulated event boundary. We then investigated whether RPE event boundaries disrupt temporal memory of those events (Exp. 4). After reward learning, we asked participants to order and estimate the distance between two events that had either included a high-RPE event between them, or not. We found (n=49) and replicated (n=77) worse sequence memory for events across a high-RPE event. Altogether, these findings demonstrate that high-RPE events are both more strongly encoded and act as event boundaries that interrupt the sequential integration of events. We captured these effects in a variant of the Context Maintenance and Retrieval model (CMR; Polyn, Norman & Kahana, 2009), modified to incorporate RPEs into the encoding process.

neuroscience↗

bioRxiv2017

Dissociable effects of surprising rewards on learning and memory

The extent to which rewards deviate from learned expectations is tracked by a signal known as a \"reward prediction error\", but it is unclear how this signal interacts with episodic memory. Here, we investigated whether learning in a high-risk environment, with frequent large prediction errors, gives rise to higher fidelity memory traces than learning in a low-risk environment. In Experiment 1, we showed that higher magnitude prediction errors, positive or negative, improved recognition memory for trial-unique items. Participants also increased their learning rate after large prediction errors. In addition, there was an overall higher learning rate in the low-risk environment. Although unsigned prediction errors enhanced memory and increased learning rate, we did not find a relationship between learning rate and memory, suggesting that these two effects were due to separate underlying mechanisms. In Experiment 2, we replicated these results with a longer task that posed stronger memory demands and allowed for more learning. We also showed improved source and sequence memory for high-risk items. In Experiment 3, we controlled for the difficulty of learning in the two risk environments, again replicating the previous results. Moreover, equating the range of prediction errors in the two risk environments revealed that learning in a high-risk context enhanced episodic memory above and beyond the effect of prediction errors to individual items. In summary, our results across three studies showed that (absolute) prediction error magnitude boosted both episodic memory and incremental learning, but the two effects were not correlated, suggesting distinct underlying systems.

animal behavior and cognition↗