A SPATIAL SAMPLING MECHANISM FOR EFFECTIVE BACKGROUND SUBTRACTION Marco Cristani, Vittorio Murino Computer Science Dep., Universit`a degli Studi of Verona, Strada Le Grazie 15, Verona, Italy cristanm@sci.univr.it, vittorio.murino@univr.it Keywords: Background subtraction, mixture of Gaussian, video surveillance Abstract: In the video surveillance literature, background (BG) subtraction is an important and fundamental issue. In this context, a consistent group of methods operates at region level, evaluating in fixed zones of interest pixel values’ statistics, so that a per-pixel foreground (FG) labeling can be performed. In this paper, we propose a novel hybrid, pixel/region, approach for background subtraction. The method, named Spatial-Time Adaptive Per Pixel Mixture Of Gaussian (S- TAPPMOG), evaluates pixel statistics considering zones of interest that change continuously over time, adopting a sampling mechanism. In this way, numerous classical BG issues can be efficiently faced: actually, it is possible to model the background information more accurately in the chromatic uniform regions exhibiting stable behavior, thus minimizing foreground camouflages. At the same time, it is possible to model successfully regions of similar color but corrupted by heavy noise, in order to minimize false FG detections. Such approach, outperforming state of the art methods, is able to run in quasi-real time and it can be used at a basis for more structured background subtraction algorithms. 1 Introduction Background subtraction is a fundamental step in automated surveillance. It represents a pixel classification task, where the classes are the back- ground (BG), i.e., the expected part of the mon- itored scene, and the foreground (FG), i.e., the interesting visual information (e.g., moving ob- jects). As witnessed by the related literature (see Sect.2), choosing the right class cannot be ade- quately performed by per pixel methods, i.e., con- sidering every temporal pixel evolution as an in- dependent process. Instead, region based meth- ods better behave, deciding the class of a pixel value by inspecting the related neighborhood. In this paper, we propose a novel approach for background subtraction which constitutes a per region extension of a widely used and effec- tive per pixel BG model, namely the Time Adap- tive Per Pixel Mixture Of Gaussian (TAPPMOG) model. The proposed approach, called Spatial- TAPPMOG (S-TAPPMOG), is based on a sam- pling mechanism, inspired by the particle filtering paradigm. The goal of the approach is to provide a per pixel characterization of the BG which takes into account selectively for contributions coming from the neighboring pixel locations. The result is constituted by a set of per pixel models which are built per region: this characterization turns out to be very robust to false FG alarms, espe- cially when the scene is heavily cluttered, and in general highly robust to the FG misses (i.e., not detected FG pixel values). In particular, several problems that classically affect BG subtraction schemes are successfully faced by the proposed method. Theoretical considerations and exten- sive comparative experimental tests prove the ef- fectiveness of the proposed approach. The rest of the paper is organized as follows. Sec- tion 2 reviews briefly the huge BG subtraction literature. In Section 3, the needed mathemati- cal fundamentals, i.e., the TAPPMOG model and the particle filtering paradigm, are reported. The whole strategy is detailed in Section 4, and, fi- nally, in Section 5, experiments on real data val- idate our method and conclude the paper.