The Role of Training Paradigms in Shaping Connectivity and Dynamics: A Comparison of Gradient-Based and Evolutionary Methods in Recurrent Neural Network Models

Abstract

Recurrent neural networks (RNNs) trained with backpropagation through time (BPTT) use gradients and global error signals to solve tasks, while evolutionary algorithms (EAs) offer an alternative solution through gradient-free optimization. Both classes of methods can solve the same tasks by modifying network weights, but it remains unclear how the choice of training paradigm biases the final connectivity structure and resulting dynamics. When drawing conclusions from task-trained RNNs, especially as proxies for neurobiological computation, it is important to consider whether the resulting network structure is due to the training method itself. Here, we compare four training paradigms--BPTT, evolution strategies (ES), genetic algorithms (GA), and GA combined with Ojas Hebbian plasticity rule (GA+Oja) -- across RNNs of 32, 64, and 128 neurons. These training paradigms were applied to two contrasting tasks that are exemplary of tasks which animals perform: a discrete working memory task and a continuous sensorimotor integration task. For both tasks, we analyzed how each training algorithm modified the network through four measurements: weight change allocations across input (Win), recurrent (Wrec), and output (Wout) layers, the effective rank of the recurrent weight matrix, the dimensionality of hidden-state dynamics, and task accuracy. BPTT achieved near-perfect accuracy across all task conditions, progressively allocated more weight changes to Wout as the working memory tasks difficulty increased, and confined hidden-state activity to a lower-dimensional subspace than any evolutionary method. Evolutionary methods maintained higher recurrent effective rank, higher activity dimensionality, and no comparable difficulty-dependent reallocation toward the readout, all while maintaining comparable task accuracy as BPTT. These findings show how gradient-based and gradient-free algorithms discover distinct structural and dynamical solutions to the same computational problems, especially in tasks involving working memory, with important implications for analyzing task-trained RNNs as models of biological neural computation. Biological neural circuits, which are shaped by evolution and local plasticity rather than gradient descent, may operate in higher-dimensional regimes than gradient-trained RNN models predict.