Internal noise promotes heterogeneous synaptic dynamics important for working memory in recurrent neural networks

Abstract

Recurrent neural networks (RNNs) based on model neurons that communicate via continuous signals have been widely used to study how cortical neurons perform cognitive tasks. Training such networks to perform tasks that require information maintenance over a brief period (i.e., working memory tasks) remains a challenge. Critically, the training process becomes difficult when the synaptic decay time constant is not fixed to a large constant number for all the model neurons. Here, we show that introducing random noise to the RNNs not only speeds up the training but also produces stable models that can maintain information longer than the RNNs trained without internal noise. Importantly, this robust working memory performance induced by internal noise during training is attributed to an increase in synaptic decay time constants of a distinct subset of inhibitory units, resulting in slower decay of stimulus-specific activity critical for memory maintenance.