Anomalous Behaviour Detection Using Spatiotemporal Oriented Energies, Subset Inclusion Histogram Comparison and Event-Driven Processing Andrei Zaharescu 1,2 and Richard Wildes 2 1 Aimetis Corporation, Waterloo, Canada 2 Department of Computer Science and Engineering, York University, Toronto, Canada {andreiz,wildes}@cse.yorku.ca Abstract. This paper proposes a novel approach to anomalous behaviour detec- tion in video. The approach is comprised of three key components. First, dis- tributions of spatiotemporal oriented energy are used to model behaviour. This representation can capture a wide range of naturally occurring visual spacetime patterns and has not previously been applied to anomaly detection. Second, a novel method is proposed for comparing an automatically acquired model of nor- mal behaviour with new observations. The method accounts for situations when only a subset of the model is present in the new observation, as when multiple activities are acceptable in a region yet only one is likely to be encountered at any given instant. Third, event driven processing is employed to automatically mark portions of the video stream that are most likely to contain deviations from the expected and thereby focus computational efforts. The approach has been imple- mented with real-time performance. Quantitative and qualitative empirical eval- uation on a challenging set of natural image videos demonstrates the approach’s superior performance relative to various alternatives. 1 Introduction Detection of anomalous behaviour relative to some model of expected behaviour is a fundamental task in surveillance scenarios. Examples include detection of movement in an area where none should occur (as in a secure storage facility) and detection of “wrong way motion” where movement of objects only should occur in one direction yet are observed in a different direction (as in movement of traffic on a one-way road). In particular, given the increase in video coverage of public and private spaces, an au- tomated ability to monitor the acquired data and signal deviations from expected be- haviour would be very useful, as it could serve to alert either human or artificial systems to analyze further the data that is acquired. A number of challenges must be surmounted for successful detection of anomalous behaviour in surveillance video. In essence, these challenges arise from the need to model a wide range of potentially complicated patterns of normal activity and detect fine deviations from that model, even while being robust to changes that are insignif- icant. Normal activity can range from simple no temporal change through single and K. Daniilidis, P. Maragos, N. Paragios (Eds.): ECCV 2010, Part I, LNCS 6311, pp. 563–576, 2010. c  Springer-Verlag Berlin Heidelberg 2010