Background recovering in outdoor image sequences: An example of soccer players segmentation Pascual J. Figueroa a , Neucimar J. Leite a, * , Ricardo M.L. Barros b a Instituto de Computac ¸a ˜o, Universidade Estadual de Campinas, Avenida Albert Einstein, 1251, Caixa Postal 6176, 13084-971 Campinas, SP, Brazil b Faculdade de Educac ¸a ˜o Fı ´sica, Laborato ´rio de Instrumentac ¸a ˜o para Biomeca ˆnica, Universidade Estadual de Campinas, Avenida E ´ rico Verı ´ssimo 701, Caixa Postal 6134, 13083-851 Campinas, SP, Brazil Received 23 September 2003; received in revised form 9 September 2005; accepted 8 December 2005 Abstract In this work, we consider the problem of background pixels information recovering which can be used, for example, in applications concerning segmentation and tracking of components in video images. Shortly, to recover the background of image sequences representing outdoor scenes, we consider a non-parametric morphological leveling operation, which takes into account the specific problem of lighting changes and the fact that we can have both slow and fast motion in the scene. We illustrate the segmentation of players based on the difference between image sequences and the corresponding recovered background representation. We also discuss the reduction of shadows in digital video of soccer games and show the good results of the whole background recovering and segmentation process. q 2006 Elsevier B.V. All rights reserved. Keywords: Background recovering; Sport video images; Mathematical morphology; Image segmentation 1. Introduction Image segmentation is an important step in many computer vision applications including, for example, tracking of moving objects, detection of intrudes in surveillance systems and recognition of people activities. For this reason, the improve- ment of algorithms for automatic segmentation has demanded a great attention of the researchers during these last years. Many aspects can make the segmentation of the interest components of an image very difficult, particularly when the events happen in an uncontrolled environment, as it is the case for sport games in outdoor scenes. In such environments, usually, we have considerable changes of illumination on a sunny day, specially in case of a cloudy sky. Beside these aspects, we also have to consider, the size and shape changes of the moving objects and their shadows, as well as the shadows of other objects projected onto the scene, which make more complex the whole segmentation process. For example, during a soccer game in the afternoon, we can see the shadows of the stadium structure moving on the playing field, together with the shadows of the players. Background subtraction is a simple and very common method used for segmenting moving objects. It consists in extracting a background image and making the difference between this background model and the current image [3]. In order to consider environmental changes, such as illumination, shadows and background objects, the background image needs to be regularly updated. For this purpose, some statistical adaptive methods have been proposed [2–4,7,9,17]. These methods update the background by modeling each pixel as a Gaussian distribution and work well in scenes with moving objects and slow lighting changes. Franc ¸ois and Medioni [4] modeled the background pixels as multi-dimensional Gaussian distributions in HSV color space. McKenna et al. [9] also considered the Gaussian distribution for modeling the back- ground pixels in RGB channels. They proposed a method, which combines the value of the RGB pixels and the chromaticity values with local image gradient. Stauffer and Grimson [17] model each pixel as a mixture of multiple Gaussians in order to consider multiple motions. Rider et al. [14] modeled each pixel based on Kalman filter. In all these methods, changes of illumination are assumed to occur slowly relative to the object motion. The background model is updated recursively using mean and variance information and the latest defined measurements. Image and Vision Computing 24 (2006) 363–374 www.elsevier.com/locate/imavis 0262-8856/$ - see front matter q 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2005.12.012 * Corresponding author. Tel.: C55 19 3788 5873; fax: C55 19 3788 5847. E-mail address: neucimar@ic.unicamp.br (N.J. Leite).