Content-based Retrieval and Real Time Detection from Video Sequences Acquired by Surveillance Systems Elena Stringa and Carlo S. Regazzoni DIBE - University of Genoa - Via Opera Pia 11A, 16145 GENOVA, ITALY stringa@dibe.unige.it carlo@dibe.unige.it Abstract In this paper, a surveillance system devoted to detect abandoned objects in unattended environments is presented to which image processing content based retrieval capabilities have been added for making easier inspection task from operators. Video-based surveillance systems generally employ one or more cameras connected to a set of monitors. This kind of systems needs the presence of a human operator, who interprets the acquired information and controls the evolution of the events in a surveyed environment. During the last years efforts have been performed to develop systems supporting human operators in their surveillance task, in order to focus the attention of operators when unusual situations are detected. Image sequences databases are also managed by the proposed surveillance system in order to provide operators with the possibility of retrieving in a second time the interesting sequences that may contain useful information for discovering causes of an alarm. Experimental results are shown in terms of the probability of correct detection of abandoned objects and examples about the retrieval sequences. 1. Introduction The most widely used video-based surveillance systems generally employ two or more monochromatic cameras that are connected to one or more monitors. This kind of systems needs the presence of a human operator, who interprets the acquired information and controls the evolution of the events in a surveyed environment. During the last years an effort has been performed to develop systems supporting human operators in their surveillance task, in order to alert the operator only when unusual situations are detected. The system proposed in this paper aims at revealing to a human operator the presence of abandoned objects in the waiting rooms of unattended railway stations (e.g. railway stations of peripheral localities, which are quite far from big urban centres). The end-users of surveillance systems have also the necessity of recovering some stored sequences in which is contained the cause of particular alarms. In order to satisfy this requirement, the proposed system has been developed in such a way that it can index image sequences. Sequence indexing is performed in order to allows one to retrieve particular sequences in a fast and efficient way; the sequences to be retrieved contain the cause of the alarm, i.e. the person who leave the object that caused the alarm (Fig. 1). The system is based on a monochromatic TV-camera acquiring video data about the surveyed environment. These data are then processed in order to alert the human operator only in case of potential dangerous situations; the same information used for the recognition task is used for indexing a particular video shot consisting in images of the scene acquired just before the alarm. This objective has been reached by subdividing the local processing system into modules, each of one implementing one of the image processing functions needed by the considered application. The modules are arranged in such a way to progressively elaborate input image sequences. The output of the system is a complex alert signal to the human operator, associated with the features of detected abandoned objects. Section 2 contains the overall description of the proposed system. In session 3 the features used for indexing the video shot are shown. Finally, the performance of the systems are shown in terms of the capability of the system of correctly recognizing abandoned objects and in term of the capability of retrieve the reason of alarms starting from the proposed indexing method. 2. A Surveillance System for Detecting Lost Objects The architecture of the proposed surveillance system is shown in Fig.2. It is composed by different level of processing, to simplify the surveillance problems. In this section, the main characteristics of the modules are presented.