Continuous variance estimation in video surveillance sequences with high illumination changes Pedro Gil-Jime ´ nez à , Hilario Go ´ mez-Moreno, Javier Acevedo-Rodrı ´guez, Saturnino Maldonado Basco ´n Dpto. de Teorı ´a de la sen˜al y Comunicaciones, Universidad de Alcala ´, 28805 Alcala ´ de Henares, Madrid, Spain article info Article history: Received 24 November 2008 Received in revised form 16 January 2009 Accepted 19 January 2009 Available online 30 January 2009 Keywords: Video surveillance Motion detection Continuous variance estimation abstract Continuous estimation of signal statistics is an important issue in many video processing systems, such as motion detection in surveillance applications. In this paper we demonstrate how results of classical expressions for variance estimation decrease in accuracy when dealing with sequences containing high illumination variations. The paper also proposes a new estimation method, and shows how, under such conditions, the accuracy of the proposed method produces better results whilst maintaining performance in scenarios with smaller changes, thus improving the motion detection stage of a video surveillance system. & 2009 Elsevier B.V. All rights reserved. 1. Introduction Background maintenance is a common block in video surveillance systems. It enables the system to update the background model, that is, some of the background statistics [1]. Frequently, the statistics used are the mean, which is an accurate representation of the background, and the variance, which provides information about the behavior of each zone of the image. For motion detection, both parameters are used together to determine whether a pixel corresponds to the background or foreground. For instance, in [2], if a pixel pðx; yÞ has a value which is more than twice its typical deviation from the mean, then that pixel is considered as belonging to the foreground, that is: Mðx; yÞ¼ Background if m xy  2s xy opðx; yÞom xy þ 2s xy Foreground otherwise ( (1) According to this, the greater the variance of a given pixel, the lower is the sensitivity, since the range of possible values for belonging to the foreground decreases. In [3], the variance is further used, along with other parameters, to classify each zone of the image according to their behavior of each zone. In this case, an error in variance computation would lead to a pixel misclassification. If we consider a video sequence, such as the one shown in Fig. 1(a), several statistics can be computed. Fig. 1(b) shows the estimated mean. As we can see, this image depicts the background of the scene, that is, an image of the scene with all moving objects removed. The variance of the sequence can also be computed, in this case shown in Fig. 2(a) or (b). Note that since the variance is not itself an image, it must be normalized in order to be visualized properly in a figure. In this case, black ðpixel ¼ 0Þ cor- responds to a variance equal to 0, and, linearly, white ðpixel ¼ 255Þ corresponds to a variance equal to 1000. In this image, we can see how pixels with high variance values (white levels in the normalized image) belong to high activity zones, for instance, the road, and low values to low activity ones. Although many more statistic can be computed, these are the two basic ones proposed in most works. However, since the input of a video surveillance system is a stream of unlimited length, these statistics need to be computed Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/sigpro Signal Processing ARTICLE IN PRESS 0165-1684/$ - see front matter & 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.sigpro.2009.01.013 à Corresponding author. E-mail address: pedro.gil@uah.es (P. Gil-Jime ´ nez). Signal Processing 89 (2009) 1412–1416