Reinforced recurrent neural networks for multi-step-ahead flood forecasts Pin-An Chen a , Li-Chiu Chang b , Fi-John Chang a,⇑ a Department of Bioenvironmental Systems Engineering, National Taiwan University, Taiwan, ROC b Department of Water Resources and Environmental Engineering, Tamkang University, Taiwan, ROC article info Article history: Received 1 February 2013 Received in revised form 2 May 2013 Accepted 21 May 2013 Available online 28 May 2013 This manuscript was handled by Andras Bardossy, Editor-in-Chief, with the assistance of Purna Chandra Nayak, Associate Editor Keywords: Reinforced real-time recurrent learning (R-RTRL) algorithm Recurrent neural network (RNN) Multi-step-ahead forecast Flood forecast summary Considering true values cannot be available at every time step in an online learning algorithm for multi- step-ahead (MSA) forecasts, a MSA reinforced real-time recurrent learning algorithm for recurrent neural networks (R-RTRL NN) is proposed. The main merit of the proposed method is to repeatedly adjust model parameters with the current information including the latest observed values and model’s outputs to enhance the reliability and the forecast accuracy of the proposed method. The sequential formulation of the R-RTRL NN is derived. To demonstrate its reliability and effectiveness, the proposed R-RTRL NN is implemented to make 2-, 4- and 6-step-ahead forecasts in a famous benchmark chaotic time series and a reservoir flood inflow series in North Taiwan. For comparison purpose, three comparative neural networks (two dynamic and one static neural networks) were performed. Numerical and experimental results indicate that the R-RTRL NN not only achieves superior performance to comparative networks but significantly improves the precision of MSA forecasts for both chaotic time series and reservoir inflow case during typhoon events with effective mitigation in the time-lag problem. Ó 2013 Elsevier B.V. All rights reserved. 1. Introduction Accurate multi-step-ahead (MSA) forecast is valuable and desired in many engineering problems, such as rainfall and flood forecasts, however it is a challenging task and difficult to achieve. A common approach to the MSA forecast is to update network parameters through online learning techniques. Online learning is a supervised machine-learning framework, which adopts the lat- est observed values to adjust model parameters for better map- pings between instances and true values in a system. Because most observational disciplines tend to infer properties of an uncer- tain system from the analysis of time-dependent data, analytical technologies for extracting the meaningful characteristics of time series data have some inherent limitations, which has been a widely discussed issue for a long time (Brockwell and Davis, 1991; Jaeger and Haas, 2004; Jothiprakash and Magar, 2012; Nair et al., 2001). Online learning algorithms have several practical and theoretical advantages such as memory-efficient implementa- tion, runtime-efficient implementation and strong guarantees on performance even in a highly variable data structure of time series (Shalev-Shwartz et al., 2004) owing to the continual receipt of true values for adjusting model parameters. Nevertheless, the main defect of online learning is ascribed to the requirement for contin- ual true values. Engineering problems frequently require models to predict many time-steps into the future without the availability of measurements in the horizon of interest. The lack of true values makes it difficult to achieve MSA forecasts. In addition, many stud- ies indicated it is not an adequate strategy to recursively adopt sin- gle-step-ahead predictions for many time-steps into the future because the errors of MSA predictors will be accumulated based on the single-step-ahead predictor (Parlos et al., 2000; Yong et al., 2010). Such time-lag problems may cause significant performance degradation when dealing with MSA forecasts for real-world applications. For the MSA streamflow forecasts during typhoon events, models with time-lag problems (i.e. no updating latest observed values) cannot keep flow trails, especially in peak flows, as the forecasting step increases. To mitigate time-lag phe- nomena occurred in online learning algorithms, it is argued whether iterative adjustments of model parameters based on addi- tional information, such as the latest true values and/or antecedent model outputs, would be beneficial to MSA forecasts. Artificial neural networks (ANNs) have the ability to approxi- mate nonlinear functions and therefore become valuable tools for various water resources problems (Cho et al., 2011; Nayak et al., 2005; Nikolos et al., 2008; Nourani et al., 2011; Nourani and Sayyah Frad, 2012). However, static neural networks might fail to establish reliable nonlinear models for predicting dynamical 0022-1694/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.jhydrol.2013.05.038 ⇑ Corresponding author. Tel.: +886 2 23639461; fax: +886 2 23635854. E-mail address: changfj@ntu.edu.tw (F.-J. Chang). Journal of Hydrology 497 (2013) 71–79 Contents lists available at SciVerse ScienceDirect Journal of Hydrology journal homepage: www.elsevier.com/locate/jhydrol