AN ON-LINE LEARNING APPROACH: A METHODOLOGY FOR TIME VARYING APPLICATIONS N.M. ROEHL * Catholic University, PUC-Rio C.P. 38063, 22452-970, R.J., Brazil and C.E. PEDREIRA Catholic University, PUC-Rio C.P. 38063, 22452-970, R.J., Brazil E-mail: pedreira@ele.puc-rio.br Abstract: In this paper a new procedure to continuously adjust weights in a multi layered neural network is proposed. The network is initially trained by using traditional Backpropagation algorithm. After this first step, non-linear programming technique is used in order to properly on line calculate the new weights sets. This methodology is tailored to be used in time varying (non-stationary) models, eliminating necessity of retraining. Numerical results for a chaotic time series and an electricity load forecasting applications are presented. Key Words: non-stationary data, neural networks, on-line training, forecasting. 1. INTRODUCTION The main challenge when dealing with non-stationary data is to balance the information related to recent data with the information previously incorporated by the model. Multi layered neural networks [1] have been successfully used in a variety of relevant problems when invariance assumptions can be properly made. An attractive feature of neural networks is its natural ability to deal with non-linear data. The procedure proposed in this paper is directed to neural non-stationary models. Few papers can be found in the literature concerning time varying connectionist models [2][3][4]. * Now at CALTECH, Pasadena, U.S.A.