IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 42, NO. 6, NOVEMBER 2012 1257 A Review of Anomaly Detection in Automated Surveillance Angela A. Sodemann, Matthew P. Ross, and Brett J. Borghetti Abstract—As surveillance becomes ubiquitous, the amount of data to be processed grows along with the demand for manpower to interpret the data. A key goal of surveillance is to detect behav- iors that can be considered anomalous. As a result, an extensive body of research in automated surveillance has been developed, of- ten with the goal of automatic detection of anomalies. Research into anomaly detection in automated surveillance covers a wide range of domains, employing a vast array of techniques. This review presents an overview of recent research approaches on the topic of anomaly detection in automated surveillance. The reviewed stud- ies are analyzed across five aspects: surveillance target, anomaly definitions and assumptions, types of sensors used and the feature extraction processes, learning methods, and modeling algorithms. Index Terms—Abnormal behavior, anomaly detection, auto- mated surveillance, behavior classification, machine learning. I. INTRODUCTION I N RECENT years, a wealth of research has been undertaken in the domain of human behavior classification in automated surveillance. Behavior classification involves the categorization or classification of perceived behavioral events by an algorithm. This research effort has been driven by an increased concern for security and safety, coupled with an overabundance of available surveillance data relative to the amount of manpower available to process it. Anomaly detection in automated surveillance is a subset of behavior classification problems reduced to a two-class or one- class classification problem. In the anomaly detection in auto- mated surveillance process, sensors in an environment collect data representing the behavior of surveillance targets, with some behaviors assumed to be anomalous. The raw sensor data are then subjected to a feature extraction process. The resulting fea- tures become the input to a modeling algorithm, in which a learning method is applied to determine the normal or anoma- lous state of the observed behavior. Fig. 1 illustrates the re- lationship between the key aspects of the process of anomaly detection in automated surveillance. The process illustrated in Manuscript received May 24, 2011; revised November 21, 2011, March 14, 2012, and June 22, 2012; accepted August 6, 2012. Date of current version De- cember 17, 2012. This work was supported by the U.S. Department of Defense. This paper was recommended by Associate Editor H. Liu. A. A. Sodemann is with the College of Technology and Innovation, Ari- zona State University, Phoenix, AZ 85069 USA (e-mail: angela.sodemann@ gmail.com). M. P. Ross is with the U.S. Air Force Academy, Colorado Springs, CO 80840 USA (e-mail: matthew.ross@us.af.mil). B. J. Borghetti is with the Air Force Institute of Technology, Dayton, OH 45433-7765 USA (e-mail: brett.borghetti@afit.edu). Digital Object Identifier 10.1109/TSMCC.2012.2215319 Fig. 1. Diagram of the flow from environment to anomaly detection, illustrating the organization of this review. Fig. 1 can be implemented either real time or offline, in which case the feature extraction and modeling is applied to recorded sensor data. This review will primarily focus on real-time im- plementations; see the review by Saykol et al. [1] for additional information specific to the offline case. Automated anomaly detection is highly useful in reducing the amount of data to be processed manually by directing attention to a specific portion of the data, to the exclusion of the vast amounts of irrelevant data. However, the problem of anomaly detection is greatly open to interpretation, and research efforts are scattered not only in approach, but also in interpretation of the problem, assumptions, and objectives. This review will at- tempt to bring synergy to these disparate efforts by evaluating the problem formulations and solution methods applied in anomaly detection research as applied to automated surveillance. Although existing surveys treat topics related to anomaly de- tection in automated surveillance, none satisfactorily treats the subject itself. The related topic of categorizing the genre of pro- duced video was covered in a review by Brezeale and Cook [2] in 2008, while the more focused topic of understanding specific events in video data was addressed in a review by Lavee et al. [3] in 2009. Buxton [4] presented a 2003 survey on understanding 1094-6977/$31.00 © 2012 IEEE