Neurocomputational underpinnings of suboptimal beliefs in recurrent neural network-based agents

Abstract

Maladaptive belief updating is a hallmark of psychiatric disorders, yet its underlying neurocomputational mechanisms remain poorly understood. While Bayesian models characterize belief updating in decision-making, they do not explicitly model neural computations or neuro-modulatory influences. To address this, we developed a recurrent neural network-based reinforcement learning framework to investigate decision-making deficits in psychiatric conditions, using schizophrenia as a test case. Agents were trained on a predictive inference task commonly used to assess cognitive deficits found in schizophrenia, including under-updating beliefs in volatile environments and over-updating beliefs in response to uninformative cues. The task thus included two conditions: (1) a change-point condition requiring adaptation in a volatile environment and (2) an oddball condition requiring resistance to outliers. We modeled these deficits by systematically manipulating key hyper-parameters associated with specific neural theories: reward prediction error (RPE) discounting and scaling (reflecting diminished dopamine responses), network dynamics disruption (reflecting impaired working memory), and rollout buffer size reduction (reflecting decreased episodic memory capacity). These manipulations reproduced schizophrenia-like decision-making impairments and revealed that suboptimal agents exhibited fewer unstable fixed points near network activity in the changepoint condition, suggesting reduced computational flexibility. This framework extends computational psychiatry by linking cognitive biases to neural dysfunction and provides a mechanistic approach to studying decision-making impairments in psychiatric disorders.