Hassabis, D. | bioRxiv Science

bioRxiv2018

Prefrontal cortex as a meta-reinforcement learning system

Over the past twenty years, neuroscience research on reward-based learning has converged on a canonical model, under which the neurotransmitter dopamine stamps in associations between situations, actions and rewards by modulating the strength of synaptic connections between neurons. However, a growing number of recent findings have placed this standard model under strain. In the present work, we draw on recent advances in artificial intelligence to introduce a new theory of reward-based learning. Here, the dopamine system trains another part of the brain, the prefrontal cortex, to operate as its own free-standing learning system. This new perspective accommodates the findings that motivated the standard model, but also deals gracefully with a wider range of observations, providing a fresh foundation for future research.

neuroscience↗

bioRxiv2017

How Cognitive and Reactive Fear Circuits Optimize Escape Decisions in Humans

Flight initiation distance (FID), the distance at which an organism flees from an approaching threat, is an ecological metric of cost-benefit functions of escape decisions. We adapted the FID paradigm to investigate how fast or slow attacking virtual predators constrain escape decisions. We show that rapid escape decisions rely on reactive fear circuits in the periaqueductal gray and midcingulate cortex (MCC), while protracted escape decisions, defined by larger buffer zones, were associated with cognitive fear circuits which include posterior cingulate cortex, hippocampus and the ventromedial prefrontal cortex, circuits implicated in strategic avoidance and behavioral flexibility. Using a Bayesian Decision Model, we further show that optimization of escape decisions under rapid flight were localized to the MCC, a region involved in adaptive motor control, while the hippocampus is implicated in optimizing decisions that update and control slower escape initiation. These results demonstrate an unexplored link between defensive survival circuits and their role in adaptive escape decisions.\n\nSignificanceHumans, like other animals, have evolved a set of circuits whose primary function is survival. In the case of predation, these circuits include reactive fear circuits involved in fast and immediate escape decisions and cognitive fear circuits that are involved in the conscious feeling of threat as well as slow strategic escape. Using neuroimaging combined with computational modeling, we support this differentiation of fear circuits by showing that fast escape decisions are elicited by the periaqueductal gray and MCC, regions involved in reactive flight. Conversely, slower escape decisions rely on the hippocampus, posterior cingulate cortex and prefrontal cortex, a circuit implicated in behavioral flexibility. These results support the role of the defensive survival circuitry in escape decisions and a separation of fear into reactive and cognitive circuits.

neuroscience↗