Learning Chaotic Dynamics under Noise with On-Line EM Algorithm Wako Yoshida, 1 Shin Ishii 1,2 Masa-aki Sato 2 1 Graduate School of Information Science, Nara Institute of Science and Technology, Ikoma, 630-0101 Japan 2 ATR Human Information Processing Research Laboratories, Kyoto, 619-0288 Japan SUMMARY In this article, we discuss the learning of chaotic dynamics by using a normalized Gaussian network (NGnet). The NGnet is trained by an on-line EM algorithm in order to learn the vector field of the chaotic dynamics. We also investigate the robustness of our approach to two kinds of noise processes: system noise and observation noise. It is shown that the trained NGnet is able to reproduce a chaotic attractor, even under the two kinds of noise. The trained NGnet also shows good prediction performance. When only part of the dynamical variables are observed, the NGnet is trained to learn the vector field in the delay coordinate space. It is shown that the chaotic dynamics is able to be learned with this method under the two kinds of noise. © 2001 Scripta Technica, Electron Comm Jpn Pt 3, 84(6): 2331, 2001 Key words: Normalized Gaussian network; on-line EM algorithm; chaotic attractor; embedding; delay coordi- nate; robustness. 1. Introduction A normalized Gaussian network (NGnet) [7] is a network of local linear regression units. This model softly partitions input space by using normalized Gaussian func- tions, and each local unit linearly approximates output within the partition. The NGnet can be interpreted as a stochastic model that has a hidden variable. We previously proposed an on-line EM algorithm for the NGnet [11]. This algorithm is effective in dynamic environments, that is, where the inputoutput distribution changes over time. For example, it is applied to an automatic control of a double pendulum in a reinforcement learning scheme [12]. Since the NGnet is a local model, it is possible to change the parameters of several units to learn a single datum. Therefore, the learning is easier than in global models such as the multilayered perceptron. In local mod- els, however, if one wants to approximate the inputoutput relationship in the whole input space, the number of neces- sary units grows exponentially as the input space dimension increases. This results in a computational explosion, which is often called the curse of dimensionality. On the other hand, real data are often distributed in a lower dimensional space than the input space, for example, attractors in dy- namical systems. In this article, we apply the NGnet and the on-line EM algorithm to an identification problem of unknown chaotic dynamics. Since a chaotic attractor usually consists of a lower-dimensional manifold than the input space, the NGnet is suitable for approximating its vector field in the neighborhood of the attractor. In our previous papers [8, 10], we showed that a recurrent neural network (RNN) can learn a chaotic dynamics. When only part of the dynamical variables were observed, the RNN was able to reproduce © 2001 Scripta Technica Electronics and Communications in Japan, Part 3, Vol. 84, No. 6, 2001 Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J83-A, No. 1, January 2000, pp. 2837 23