Background Subtraction and Shadow Detection in Grayscale Video Sequences Julio Cezar Silveira Jacques Jr Cl´ audio Rosito Jung Soraia Raupp Musse CROMOS Laboratory - PIPCA University of Vale do Rio dos Sinos Av. Unisinos 950, 93022-000 S˜ ao Leopoldo, RS, Brazil phone: +55 51 5908161 and fax: +55 51 5908162 julioj@turing.unisinos.br, crjung@unisinos.br, soraiarm@exatas.unisinos.br Abstract Tracking moving objects in video sequence is an impor- tant problem in computer vision, with applications several fields, such as video surveillance and target tracking. Most techniques reported in the literature use background sub- traction techniques to obtain foreground objects, and ap- ply shadow detection algorithms exploring spectral infor- mation of the images to retrieve only valid moving objects. In this paper, we propose a small improvement to an existing background model, and incorporate a novel technique for shadow detection in grayscale video sequences. The pro- posed algorithm works well for both indoor and outdoor sequences, and does not require the use of color cameras. 1 Introduction A relevant problem in computer vision is the detection and tracking of moving objects in video sequences. Possible applications include surveillance [6, 7, 12], traffic monitor- ing [8] and athletic performance analysis [1], among others. In applications using fixed cameras with respect to the static background (e.g. stationary surveillance cameras), a very common approach is to use background subtrac- tion to obtain an initial estimate of moving objects. Basi- cally, background subtraction consists of comparing each new frame with a representation of the scene background: significative differences usually correspond to foreground objects. Ideally, background subtraction should detect real moving objects with high accuracy, limiting false negatives (objects pixels that are not detected) as much as possible; at the same time, it should extract pixels of moving objects with the maximum responsiveness possible, avoiding de- tection of transient spurious objects, such as cast shadows, static objects, or noise. In particular, the detection of cast shadows as fore- ground objects is very common, producing undesirable con- sequences. For example, shadows can connect different people walking in a group, generating a single object (typ- ically called blob) as output of background subtraction. In such cases, it is more difficult to isolate and track each per- son in the group. There are several techniques for shadow detection in video sequences [2–4, 10, 13, 15–17], and the vast major- ity of them are based on color video sequences. Although color images indeed provide more information for shadow detection, there are still several scenarios where monochro- matic video cameras are utilized In this paper, we improve the background subtraction technique described in [6], and propose a new shadow detection algorithm for grayscale im- ages. The remainder of this paper is organized as follows. Sec- tion 2 presents related work concerning background sub- traction and shadow detection. The proposed technique is described in Section 3, and some experimental results are provided in Section 4. Finally, conclusions are given in Sec- tion 5. 2 Related Work Several techniques for background subtraction and shadow detection have been proposed in the past years. Background detection techniques may use grayscale or color images, while most shadow detection methods make use of chromaticity information. Next, some of these tech- niques are described. The car tracking system of Koller et al. [9] used an adap- tive background model based on monochromatic images fil- tered with Gaussian and Gaussian derivative (vertical and