Learning techniques for structured networks

Abstract

A convergence analysis is presented for the training of structured networks. Since the learning techniques that are used in structured networks are the same as the ones used in training of neural networks, the issue of convergence is discussed not only from a numerical perspective but also as a means of deriving insight into connectionist learning. In the analysis, bounds are developed on the learning rate, under which exponential convergence of the weights to their correct values is proved for a class of matrix algebra problems that includes linear equation solving, matrix inversion, and Lyapunov equation solving.