Abnormal Detection Using Interaction Energy Potentials Xinyi Cui, Qingshan Liu, Mingchen Gao, Dimitris N. Metaxas Department of Computer Science, Rutgers University, Piscataway, NJ, USA {xycui, qsliu, minggao, dnm}@cs.rutgers.edu Abstract A new method is proposed to detect abnormal behaviors in human group activities. This approach effectively models group activities based on social behavior analysis. Differ- ent from previous work that uses independent local features, our method explores the relationships between the current behavior state of a subject and its actions. An interaction energy potential function is proposed to represent the cur- rent behavior state of a subject, and velocity is used as its actions. Our method does not depend on human detection or segmentation, so it is robust to detection errors. Instead, tracked spatio-temporal interest points are able to provide a good estimation of modeling group interaction. SVM is used to find abnormal events. We evaluate our algorithm in two datasets: UMN and BEHAVE. Experimental results show its promising performance against the state-of-art methods. 1. Introduction Abnormal event detection plays an important role in video surveillance and smart camera systems. Various ab- normal activities have been studied, including restricted- area access detection [9], car counting [6], detection of peo- ple carrying cases [7], abandoned objects [22], group activ- ity detection [31, 17], social network modeling[29], mon- itoring vehicles [28], scene analysis [24] and so on. In this paper, we focus on modeling abnormal events in hu- man group activities, which is a very important application for video surveillance. Fig.1 shows two sample frames. (a) shows a group of people fighting in the street, and (b) shows people running away from the scenes. We propose a new method to detect abnormal events in group activities. We represent group activities by learning relationships between the current behavior state of a sub- ject and its actions. Our goal is to explore the reasons why people take different actions under different situations. In the real world, people are driven by their goals. They take into account of the environment as well as the influ- ence of other people. We define an interaction energy po- tential function to represent the current state of a subject based on the positions/velocities of a subject itself as well as its neighbors. Fig.2 shows an example of interaction en- ergy potentials and velocities. Section 2 gives the details of the definition. Social behaviors are captured by the rela- tionship between interaction energy potential and its action, which is then used to describe social behaviors. Uncommon Energy-Action patterns indicate an abnormal activity. Ex- periments on two datasets UMN [1] and BEHAVE [2] show that our method is powerful to model abnormal behaviors in group activities. Our contributions. 1) The Interaction Energy Poten- tial is proposed to model the relationship among a group of people. 2) The relationship between the current state of a subject and the corresponding reaction is explored to model the normal/abnormal patterns. 3) Our method does not rely on human detection or segmentation technique, so it is more robust to the errors that are introduced by detec- toin/segmentation techniques. (a) (b) Fighting Panic Figure 1. Abnormal event examples. (a) a group of people fighting; (b) People are panic, trying to run away from the scene. 1.1. Related Work Human action/activity modeling in video sequences is a hot topic in the communities of computer vision and pat- tern recognition. In recent years, a lot of algorithms have 3161