Modeling and prediction with a class of time delay dynamic neural networks § Yasar Becerikli a, * , Yusuf Oysal b a Department of Computer Engineering, Kocaeli University, Izmit, Turkey b Anadolu University, Computer Engineering Department, Eskisehir, Turkey Available online 28 February 2006 Abstract In this paper, we propose a time delay dynamic neural network (TDDNN) to track and predict a chaotic time series systems. The application of artificial neural networks to dynamical systems has been constrained by the non-dynamical nature of popular network architectures. Many of the drawbacks caused by the algebraic structures can be overcome with TDDNNs. TDDNNs have time delay elements in their states. This approach provides the natural properties of physical systems. The minimization of a quadratic performance index is considered for trajectory tracking applications. Gradient computations are presented based on adjoint sensitivity analysis. The computational complexity is significantly less than direct method, but it requires a backward integration capability. We used Levenberg–Marquardt parameter updating method. # 2006 Elsevier B.V. All rights reserved. Keywords: Dynamic neural networks; Time delay; Attractor; Chaos; Tracking trajectory; Prediction; Adjoint theory 1. Introduction Chaotic time series are considered as the outputs of nonlinear dynamic systems. If one cannot specify the initial condition with infinite precision, the long time future behavior of these time series is unpredictable. But, the short time behavior can be exactly encapsulated. Many types of time delays are observed such as axonal propagation delays and synaptic transmission delays in biological neural networks. There are various types of neural networks with time delays. For solving time-sequence recognition problem, the delayed synaptic connections were used in some neural networks [1,2]. Such time-delay neural networks (TDNNs) have been widely used in some practical engineering problem such as nonlinear predictions and recogni- tion. Hebbian-type neural network was based as delayed feedback connections [3,4]. Stability analysis of TDNNs has been extensively analyzed in many studies [5–9]. Our focus in this work is to model and predict a nonlinear system with time delay by using time delay dynamic neural networks (TDDNNs) with fast supervised training algorithms such as Levenberg–Marquardt algorithm [10,11]. The TDDNN stands for a continuous-time recurrent neural network that has time-delayed feedbacks. The gradient algorithm is based on adjoint theory [12–17] that is faster than the forward method. In Section 2, we present a model for time-delay dynamic neural networks (TDDNNs) and describe the class of applications we have considered—trajectory tracking and prediction. Some illustrative examples are given in Section 3. Given a desired trajectory, a nonlinear optimization problem must be solved to determine appropriate values for network parameters, and we have employed gradient-based approaches, discussed in Section 4. Chaotic time series prediction experimental results are presented in Section 5. 2. The time delay dynamical neural network model structure The dynamical system considered describes an electronic circuit of n saturable amplifiers (called neurons) coupled by a resistive interconnection matrix (that is weights). This can be contrasted three simpler network architectures: (i) In which connectivity is constrained to be feedforward and there are no dynamics in the processing units. Feedforward/algebraic net- works are the workhorses of neural network applications. (ii) In which feedback connections are allowed but dynamics in processing units are not. Networks of this type have been used for several symbolic processing tasks [18–20]. (iii) In which the connectivity is constrained to be feedforward but dynamical and delay elements are included. For example, the output of the unit www.elsevier.com/locate/asoc Applied Soft Computing 7 (2007) 1164–1169 § Initial version of this paper has been partially presented on the ‘‘12th Mediterranean Conference on Control and Automation MED’04, June 6–9, 2004, Kusadası, Turkey’’. * Corresponding author. Tel.: +90 262 3351168; fax: +90 262 3351150. E-mail addresses: becer@kou.edu.tr, ybecer@ieee.org (Y. Becerikli). 1568-4946/$ – see front matter # 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.asoc.2006.01.012