Combining Neural Networks and Fuzzy Systems for Human Behavior Understanding Giovanni Acampora School of Industrial Engineering, Information Systems, Eindhoven University of Technology (the Netherlands) g.acampora@tue.nl Pasquale Foggia, Alessia Saggese, Mario Vento Department of Electronic and Information Engineering (DIEII) University of Salerno (Italy) {pfoggia, asaggese, mvento}@unisa.it Abstract The psychological overcharge issue related to human in- adequacy to maintain a constant level of attention in simul- taneously monitoring multiple visual information sources makes necessary to develop enhanced video surveillance systems that automatically understand human behaviors and identify dangerous situations. This paper introduces a semantic human behavioral analysis (HBA) system based on a neuro-fuzzy approach that, independently from the spe- cific application, translates tracking kinematic data into a collection of semantic labels characterizing the behavior of different actors in a scene in order to appropriately classify the current situation. Different from other HBA approaches, the proposed system shows high level of scalability, robust- ness and tolerance for tracking imprecision and, for this reason, it could represent a valid choice for improving the performance of current systems. 1. Introduction The rapid growth of the video surveillance market asso- ciated with the increasing need to secure people and goods is contributing to an exponential decrease of the price of monitoring sensors (cameras, microphones and so on) and, as a consequence, it is leading to a scenario where a great amount of visual material needs to be monitored and ana- lyzed in real time in order to detect and face possible dan- ger or emergency situations. However, human beings show limited capabilities in performing this task due to their in- adequacy to maintain a constant level of attention [1] [6] in simultaneously monitoring multiple screens or other visual information sources. In order to try to overcome this psy- chological overcharge issue [12], computer vision and arti- ficial intelligence techniques can support and enhance the design and development of innovative video surveillance systems capable of understanding the content of video se- quences, classifying human activities and identifying abnor- mal situations. In general, this goal is achieved by defining a processing scheme based on the following macro-stages: human tracking and human behavioral analysis. The first stage is devoted to detect the human motion and compute a collection of kinematic information related to the different people populating a scene, whereas the aim of the second stage is to analyze the kinematic information and to iden- tify spatio-temporal human activities so as to understand the behaviors and relationships among different people in the scene. Although the tracking problem cannot be con- sidered definitively solved, there are in the literature many algorithms [5] [3] [4] [2] that attempt to provide reasonably usable solutions; these algorithms have been used for a sig- nificant time and have received an assessment and a charac- terization of their limits and their applicability. On the other hand, human behavioral analysis represents an embryonic and challenging research topic that has been attracted the attention of different scientists groups since the few years. Indeed, different human behavioral analysis systems, based on different computational approaches, have been recently proposed. For instance, the ADVISOR project [11] exploits a description logic language for trying to detect anoma- lous situations in complex scenes opportunely represented by means of three-dimensional models. Recently, Mecocci et al. [10] presented an architecture for real-time video surveillance capable of autonomously detecting anomalous events by comparing human trajectories with a collection of typical learned prototypes. The OBSERVER [7] project 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance 978-0-7695-4797-8/12 $26.00 © 2012 IEEE DOI 10.1109/AVSS.2012.25 88