Pre-training artificial neural networks with spontaneous retinal activity improves motion prediction in natural scenes

Abstract

The ability to process visual stimuli rich with motion represents an essential skill for animal survival and is largely already present at the onset of vision. Although the exact mechanisms underlying its maturation remain elusive, spontaneous activity patterns in the retina, known as retinal waves, have been shown to contribute to this developmental process. Retinal waves exhibit complex spatio-temporal statistics and contribute to the establishment of circuit connectivity and function in the visual system, including the formation of retinotopic maps and the refinement of receptive fields in downstream areas such as the thalamus and visual cortex. Recent work in mice has shown that retinal waves have statistical features matching those of natural visual stimuli, such as optic flow, suggesting that they could prime the visual system for motion processing upon vision onset. Motivated by these findings, we examined whether artificial neural network (ANN) models trained on natural movies show improved performance if pre-trained with retinal waves. We employed the spatio-temporally complex task of next-frame prediction, in which the ANN was trained to predict the next frame based on preceding input frames of a movie. We found that pre-training ANNs with retinal waves enhances the processing of real-world visual stimuli and accelerates learning. Strikingly, when we merely replaced the initial training epochs on naturalistic stimuli with retinal waves, keeping the total training time the same, we still found that an ANN trained on retinal waves temporarily outperforms one trained solely on natural movies. Similar to observations made in biological systems, we also found that pre-training with spontaneous activity refines the receptive field of ANN neurons. Overall, our work sheds light on the functional role of spatio-temporally patterned spontaneous activity in the processing of motion in natural scenes, suggesting it acts as a training signal to prepare the developing visual system for adult visual processing. Author summaryBefore the onset of vision, the retina generates its own spontaneous activity, referred to as retinal waves. This activity is crucial for establishing neural connections and, hence, ensuring the proper functionality of the visual system. Recent research has shown that retinal waves exhibit statistical properties similar to those of natural visual stimuli, such as the optic flow of objects in the environment during forward motion. We investigate whether retinal waves can prepare the visual system for motion processing by pre-training artificial neural network (ANN) models with retinal waves. We tested the ANNs on next-frame prediction tasks, where the model predicts the next frame of a video based on previous frames. Our results showed that ANNs pre-trained with retinal waves exhibit faster learning on movies featuring naturalistic stimuli. Additionally, pre-training with retinal waves refined the receptive fields of ANN neurons, similar to processes seen in biological systems. Our work highlights the importance of spatio-temporally patterned spontaneous activity in preparing the visual system for motion processing in natural scenes.