Combining Degrees of Normality Analysis in Intelligent Surveillance Systems J. Albusac Faculty of Engineering Almad´ en (Spain) University of Castilla-la Mancha Email: JavierAlonso.Albusac@uclm.es D. Vallejo and L. Jim´ enez Faculty of Computing Ciudad Real (Spain) University of Castilla-la Mancha J.J. Castro-Schez and C. Glez-Morcillo Faculty of Computing Ciudad Real (Spain) University of Castilla-la Mancha Abstract—Advanced Surveillance Systems are able to automat- ically understand events and behaviors. These systems carry out an exhaustive analysis from multi-sensor information, according to multiple aspects or events of interest in order to classify situations as normal or abnormal. Thus, developing appropriate methods in order to combine the information from several criteria becomes critical to achieve a reliable interpretation in monitored environments. In this paper, we address the aggregation problem for multiple criteria in the domain of intelligent surveillance and analyze several alternatives to be put in practice. From these alternatives, we also propose a new aggregation method based on the Sugeno integral. All these methods have been evaluated within the context of OCULUS, an intelligent surveillance system that has been used to successfully monitor trajectories and speed of moving objects. I. I NTRODUCTION Intelligent Surveillance Systems (ISS) aim at automatically understanding events and object behaviors. These systems tend to replace the traditional surveillance systems which are composed of a set of security cameras that send the video signal to a central room where the security staff monitor the environment. Precisely, the security staff have to continuously analyze the images captured by the cameras to check that everything works according to the established normality in the monitored environment. However, human agents are affected by negative factors such as fatigue or tiredness after prolonged period of observation. Thus, one of the goals of intelligent surveillance lies in overcoming these limitations and improv- ing some of the tasks carried out by the human agents. Usually, complex monitoring involves the analysis of het- erogeneous information obtained from multiple kinds of sources such a cameras, microphones, infrared sensors and so on. From this information, together with a knowledge base that describes normal behaviors and events, the artificial system is able to track and classify moving objects, interpreting their behavior according to different aspects or events of interest such as trajectories, speed, abandoned objects, etc. ISS try to understand events in a similar way as human beings do when monitoring complex environments. Within this context, many individual events and the relationships among them are considered to get a global evaluation of the currently monitored situation. At this point, it is important to remark that there are situations or events which do not imply danger when they occur in isolation but should be seriously taken into account when they take place simultaneously. Thus, the behavior understanding of moving objects do not only depend on one specific aspect but a combination of multiple ones. So, in order to design more and more sophisticated ISS, closer to the skills of the security staff, it is necessary to design artificial systems capable of combining multiple criteria to get a general idea about how an object behaves. This is essential to provide the surveillance systems with the ability of correctly classifying behaviors so that they can successfully assist the security guards. In previous works, we developed OCULUS [1], an intelli- gent surveillance system based on normality components [2]. A normality component specifies how an object must ideally behave according to a surveillance aspect such as trajectories or speed. A system developed under this approach scales easily and increases its analytical skills as new normality components are designed and integrated into the artificial system. We adopted the term degree of normality to refer to the output of each normality component, which represents how normal an object behaves according to a specific surveillance aspect. However, the combination of multiple normality values poses a new challenge when monitoring individual objects and complex environments. How should the global normality value of monitored objects be calculated from these criteria? How normal is the global behavior of these objects? To answer these questions, a flexible approach that considers the relationships between normality components and their importance, from a general point of view, must be devised. In other words, an abnormal situation detected by a component should significantly affect the global normality value, although the component was initially weighted with a low value. Thus, it is necessary not only to devise a method that establishes the degree of importance of each component but also to define mechanisms to dynamically adapt those data to the output values. The final normality value should be the result of combining multiple criteria whose weights may change depending on the current analyzed situations. The aim of this paper is to address this problem (Section III) and analyze several methods to combine the output values of normality components (Section IV). 2436