IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 23, NO. 8, AUGUST 2012 1269 Reinforced Two-Step-Ahead Weight Adjustment Technique for Online Training of Recurrent Neural Networks Li-Chiu Chang, Pin-An Chen, and Fi-John Chang Abstract—A reliable forecast of future events possesses great value. The main purpose of this paper is to propose an innovative learning technique for reinforcing the accuracy of two-step- ahead (2SA) forecasts. The real-time recurrent learning (RTRL) algorithm for recurrent neural networks (RNNs) can effectively model the dynamics of complex processes and has been used successfully in one-step-ahead forecasts for various time series. A reinforced RTRL algorithm for 2SA forecasts using RNNs is proposed in this paper, and its performance is investigated by two famous benchmark time series and a streamflow during flood events in Taiwan. Results demonstrate that the proposed reinforced 2SA RTRL algorithm for RNNs can adequately forecast the benchmark (theoretical) time series, significantly improve the accuracy of flood forecasts, and effectively reduce time-lag effects. Index Terms— Real-time recurrent learning (RTRL) algorithm, recurrent neural network (RNN), streamflow forecast, time series forecast. I. I NTRODUCTION M OST observational disciplines tend to infer proper- ties of an uncertain system from the analysis of its measured data. The analytical technologies for extracting the meaningful characteristics of time series data have been widely discussed for a long time [1]. Many mature tech- niques associated with time series analysis were used in many important applications such as environment and marketing [2]. Because observations closer together in time generally would be more closely related than observations further apart, it is more difficult to obtain a satisfactory multistep-ahead forecast. The recursive use of one-step-ahead forecasts for many time steps into the future is a commonly used strategy, which unfortunately has been shown to have shortcomings in real- world applications [3]. This is mainly because a small forecast error at the beginning could propagate into the future. To solve such a problem, it is argued whether an iterative adjustment of Manuscript received September 2, 2011; revised May 6, 2012; accepted May 10, 2012. Date of publication June 14, 2012; date of current version July 16, 2012. This work was supported in part by the National Science Council, Taiwan, under Grant 100-2313-B-002-011-MY3. L.-C. Chang is with the Department of Water Resources and Environmental Engineering, Tamkang University, New Taipei City 25137, Taiwan (e-mail: changlc@mail.tku.edu.tw). P.-A. Chen and F.-J. Chang are with the Department of Bioenvironmental Systems Engineering, National Taiwan University, Taipei 10617, Taiwan (e-mail: f98622003@ntu.edu.tw; changfj@ntu.edu.tw). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNNLS.2012.2200695 the model’s parameters based on additional information, such as antecedent observed values and/or model outputs, would be beneficial to multistep-ahead forecasts. Over the last few decades, artificial neural networks (ANNs) have been recognized for modeling the underlying nonlin- earities and complexities in artificial or physical systems. Many ANNs were developed to solve different problems, such as rainfall and streamflow forecasting [4]–[9], seismic [10], reservoir flood control [11], financial forecasts [12], sunspot activity [13], and many other disciplines for multistep- ahead forecasts [13]–[17]. Most of these applied with neural networks are classified into static neural networks and can simulate the short-term memory structures within processes, whereas the extraordinary time variation characteristics of time series might not be well retained. Lately, recurrent neural networks (RNNs) have attracted much attention [18]–[24] for extracting dynamic time variation characteristics. Because of their dynamic nature, RNNs have been successfully applied to a wide variety of problems such as system identification [25]–[27], speech processing and plant control [28], and time series forecasting [29]–[33]. RNNs are capable of improving forecast accuracy [3], [34]–[36]. The training of an RNN, however, could be time consuming [13], [37], such as back-propagation through time (BPTT). BPTT, designed for training RNNs, can be derived by unfolding the temporal operation of the network into a multilayer feedfor- ward network. The two familiar implementations of BPTT are the batch mode (epoch-wise BPTT) and real-time mode (truncated BPTT) [38]. A potential drawback of truncated BPTT is that the memory effects exceeding the truncation depth (duration) cannot be captured by RNNs. The real-time recurrent learning (RTRL) algorithm, proposed by Williams and Zipser [39], is an effective and efficient algorithm for training recurrent networks, and its name is derived from the fact that real-time adjustments are made to the synaptic weights of an RNN. A num- ber of previous studies demonstrated that the RTRL algo- rithm for RNNs is very effective in modeling the dynamics of complex processes and can provide accurate forecasts [40]–[42], while some studies further made efforts to reduce the time complexity of the RTRL algorithm [43]–[45]. Due to geophysical conditions, reservoirs in Taiwan are relatively small when considering the amount of water falling on watersheds during typhoon events. A controlled spillway is equipped with mechanical gates to control the water release 2162–237X/$31.00 © 2012 IEEE