Sequential prediction of unbounded stationary time series L´ aszl´ o Gy¨ orfi, Fellow, IEEE, and Gy¨ orgy Ottucs´ ak Abstract A simple on-line procedure is considered for the prediction of a real valued sequence. The algorithm is based on a combination of several simple predictors. If the sequence is a realization of an unbounded stationary and ergodic random process then the average of squared errors converges, almost surely, to that of the optimum, given by the Bayes predictor. An analog result is offered for the classification of binary processes. Index Terms On-line learning, sequential prediction, time series, universal consistency, pattern recognition. I. I NTRODUCTION We study the problem of sequential prediction of a real valued sequence. At each time instant t = 1, 2,..., the predictor is asked to guess the value of the next outcome y t of a sequence of real numbers y 1 ,y 2 ,... with knowledge of the pasts y t−1 1 =(y 1 ,...,y t−1 ) (where y 0 1 denotes the empty string) and the side information vectors x t 1 =(x 1 ,...,x t ), where x t ∈ R d . Thus, the predictor’s estimate, at time t, is based on the value of x t 1 and y t−1 1 . A prediction strategy is a sequence g = {g t } ∞ t=1 of functions g t :  R d  t × R t−1 → R so that the prediction formed at time t is g t (x t 1 ,y t−1 1 ). In this paper we assume that (x 1 ,y 1 ), (x 2 ,y 2 ),... are realizations of the random variables (X 1 ,Y 1 ), (X 2 ,Y 2 ),... such that {(X n ,Y n )} ∞ −∞ is a jointly stationary and ergodic process. L. Gy¨ orfi is with Department of Computer Science and Information Theory Budapest University of Technology and Economics, Magyar tud´ osok k¨ or´ utja 2., Budapest, Hungary, H-1117 (email: gyorfi@szit.bme.hu). Gy. Ottucs´ ak is with Department of Computer Science and Information Theory Budapest University of Technology and Economics, Magyar tud´ osok k¨ or´ utja 2., Budapest, Hungary, H-1117 (email: oti@szit.bme.hu). February 1, 2007 DRAFT