bioRxiv · 10.1101/2023.03.14.532617
Rats rapidly switch between retrospective and inferential value computations
Abstract
The value of the environment determines animals motivational states and sets expectations for error-based learning1-3. How are values computed? Rein-forcement learning systems can store or "cache" values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single tri-als. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for re-wards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environ-ment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behav-iors.
Source connections
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Mah, A., Bossio, V., Constantinople, C. M.. 2023-03-15. Rats rapidly switch between retrospective and inferential value computations. https://doi.org/10.1101/2023.03.14.532617
Cite the original work for its findings. Save a collection to share your selection of sources.