Competition between memories for reactivation as a mechanism for long-delay credit assignment

Abstract

Animals can associate events with their outcomes, even if there is a long delay between the two. For example, in conditioned taste aversion, animals gain an aversion to a taste (the conditioned stimulus, CS) if sickness (an unconditioned stimulus, US) is induced up to 12 hours later. Established correlational plasticity mechanisms, operating on timescales of milliseconds to seconds, do not wholly explain how networks of neurons achieve such long-delay credit assignment. Moreover, if the animal experiences an intervening taste (an interfering stimulus, IS), the IS gains some "credit" for the causality of the outcome, reducing aversion to the CS. We hypothesize that reactivation of prior events at the time of outcome causes specific associative learning between those events and the outcome. We explore the inherent competition underlying credit assignment using a spiking neural network model storing memories through time-decaying synaptic strengthenings in two groups of neurons producing inherently competing attractor states. We show how the later memory can be reactivated more often and reduce the reactivation of a prior memory. Also, we provide a mechanism for the experimental finding of a rebound in association with, and therefore aversion to, the CS if the time between the following IS and US is increased. Such a result can appear paradoxical as associations typically diminish with time, but arises when the IS initially produces a strong decrease in reactivation of the CS, but reactivations of the CS thereafter increase, in spite of weakening synaptic strengths, because competing reactivations of the IS decrease more. By reactivating the memories probabilistically, neural circuits can assign the credit in a biologically plausible way.