bioRxiv · 10.1101/2022.03.01.482493
A reduced-memory multi-layer perceptron withsystematic network weights generated and trainedthrough distribution hyper-parameters
Abstract
A multi-layer perceptron (MLP) consists of a number of forward-connected weights (Wijk) from each feeding layer node (nij) to the many initially equivalent nodes (ni+1,k) in the next layer. Exact a priori order and search space of these weights (Wijk) is random and prone to redundancy, irreproducibility and non-optimality. We demonstrate that a weight subspace (Wijk for each i and j), generated systematically using a statistical distribution with predefined breakpoints and Genetic algorithm-trained hyper-parameters substantially reduces the computational complexity of an MLP and produces comparable or better performance than similarly trained equivalent models with fully defined weights. This distribution based neural network (DBNN) provides a novel framework to create very large neural network models with currently prohibitive memory requirements.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Ahmad, S., Vinayak, N.. 2022-03-03. A reduced-memory multi-layer perceptron withsystematic network weights generated and trainedthrough distribution hyper-parameters. https://doi.org/10.1101/2022.03.01.482493
Cite the original work for its findings. Save a collection to share your selection of sources.