Monitoring Human Behavior in an Assistive Environment using multiple Views Dimitrios Kosmopoulos Institute of Informatics and Telecommunications NCSR Demokritos 15310, Agia Paraskevi, Greece dkosmo@iit.demokritos.gr Panagiota Antonakaki Institute of Informatics and Telecommunications NCSR Demokritos 15310, Agia Paraskevi, Greece ganton@iit.demokritos.gr Konstandinos Valasoulis Institute of Informatics and Telecommunications NCSR Demokritos 15310, Agia Paraskevi, Greece kvalas@iit.demokritos.gr Dimitrios Katsoulas Institute of Informatics and Telecommunications NCSR Demokritos 15310, Agia Paraskevi, Greece dkats@iit.demokritos.gr ABSTRACT This paper presents our approach in understanding the be- havior of humans moving on a plane using multiple cam- eras. This approach is appropriate for monitoring people in an assistive enivronment for the purpose of issuing alerts in cases of abnormal behavior. We perform camera reg- istration based on homography estimation and we extract position on 2D projection map. We use the output of mul- tiple classifiers to model and extract abnormal behaviour from both the target trajectory and the target short term activity (i.e., walking, running, abrupt motion etc). The proposed approach is verified experimentally in an indoor environment. The experiments are performed with a single moving target, however the method can be generalised to multiple moving targets, which may occlude each other, due to the use of multiple cameras. Categories and Subject Descriptors I.4.9 [Image Processing and Computer Vision]: Ap- plications—video analysis ; I.2.10 [Artificial Intelligence]: Vision and Scene Understanding—motion ; I.5.4 [Pattern Recognition]: Applications—computer vision Keywords behavior monitoring, homography, optical flow, Hidden Markov Model 1. INTRODUCTION Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Petra ’2008 Athens, Greece Copyright 2008 ACM 978-1-60558-067-8-15/07/08 ...$5.00. One of the key questions in creating pervasive systems for the care of elderly is the ”graceful integration with the human user”’ [17]. Computer vision lends itself as a very appealing method due to the fact that it is non intrusive. The main challenge in this case is to transform the video stream into a useful source of information. The main prob- lems in that case are how to track people in the captured video stream, how to identify and label individuals and how to analyze their behaviors. In this paper we are going to deal mainly with the third problem. Motion analysis in video, and particularly human behaviour understanding, has attracted many researchers [12], mainly because of its fundamental applications in video surveillance, video indexing, virtual reality and computer-human inter- faces. One of the most challenging problems in computer vision is to automatically model and recognize human be- haviour, thus reducing human intervention. Such a system will monitor a room and will automatically detect, cate- gorize and recognize human behaviours, calling for human attention only when necessary. The research in the area of behaviour understanding con- centrates mainly on the development of methods for analysis of visual data in order to extract and process information about the behavior of actors in a scene. Many methods have been proposed, as we present in the next paragraph, the common problems are occlusions in crowded scenes and lack of well defined spatial information in case of 2D track- ing. In case of 3D tracking through multiple cameras the problem is the high system complexity. The goal of our method is to approach the problem by modelling the normal and the abnormal behaviours. Since multiple criteria can be taken into consideration when an observer is trying to decide whether an observed behaviour is normal or not, such as rare activities, or abnormal sequence of usual activities, or unusual trajectories, we implemented a system that applies these criteria. To overcome problems like occlusion, view variance and high complexity we have developed a system with multi-