Comparison of Four Neural Net Learning Methods for Dynamic System Identification

Nonlinear system identification is being reexamined by many researchers using neural networks. Although both batch learning and pattern learning are employed at present, it is not clear yet whether they will always give rise to the same model. Further, validity of pattern learning has not been given in a strict mathematical sense, although it is practically effective. In this paper, in order to establish a groundwork for comparison, both pattern learning and batch learning rules are rederived for nonlinear system identification using multilayer feedforward networks and external recurrent networks. Furthermore, the conditions under which the pattern learning rules will give approximately the same model as the batch learning rules are derived as three theorems. Pattern learning is shown to approach batch learning provided that the learning rate is small. As a result, the validity of pattern learning is established. Finally, theoretical results are illustrated via simulation. © 1992 IEEE