KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS VOL. 12, NO. 8, Aug. 2018 3769 Copyright ⓒ 2018 KSII Social Pedestrian Group Detection Based on Spatiotemporal-oriented Energy for Crowd Video Understanding Shaonian Huang 1 , Dongjun Huang 2 , Mansoor Ahmed Khuhroa 2 1 Key Laboratory of Hunan Province for New Retail Virtual Reality Technology, School of Computer and Information Engineering, Hunan University of Commerce Changsha, Hunan, China [e-mail :hsn@hunnu.edu.cn] 2 School of Information Science and Engineering, Central South University Changsha, Hunan, China [e-mail:djhuang@csu.edu.cn, khuhro.mansoor@csu.edu.cn] *Corresponding author: Shaonian Huang Received February 25, 2017; revised August 31, 2017; accepted April 6, 2018; published August 31, 2018 Abstract Social pedestrian groups are the basic elements that constitute a crowd; therefore, detection of such groups is scientifically important for modeling social behavior, as well as practically useful for crowd video understanding. A social group refers to a cluster of members who tend to keep similar motion state for a sustained period of time. One of the main challenges of social group detection arises from the complex dynamic variations of crowd patterns. Therefore, most works model dynamic groups to analysis the crowd behavior, ignoring the existence of stationary groups in crowd scene. However, in this paper, we propose a novel unified framework for detecting social pedestrian groups in crowd videos, including dynamic and stationary pedestrian groups, based on spatiotemporal-oriented energy measurements. Dynamic pedestrian groups are hierarchically clustered based on energy flow similarities and trajectory motion correlations between the atomic groups extracted from principal spatiotemporal-oriented energies. Furthermore, the probability distribution of static spatiotemporal-oriented energies is modeled to detect stationary pedestrian groups. Extensive experiments on challenging datasets demonstrate that our method can achieve superior results for social pedestrian group detection and crowd video classification. Keywords: Pedestrian group detection, spatiotemporal-oriented energy, crowd video analysis, video classification http://doi.org/10.3837/tiis.2018.08.012 ISSN : 1976-7277