MobiRAR: Real-Time Human Activity Recognition Using Mobile Devices Cuong Pham Computer Science Department, Posts and Telecommunications Institute of Technology, Hanoi, Vietnam cuongpv@ptit.edu.vn Abstract—In this paper we present MobiRAR, a real-time human activity recognition system using mobile devices. The system utilizes the acceleration sensing data from the accelerometer commonly instrumented in mobile devices such as smart phones or smart watches. Our activity recognition method comprises of four steps: data processing, segmentation, feature extraction, and classification. Particularly, the set of features extracted from acceleration sensing data is invariant to device rotations. The proposed method is rigorously evaluated through a dataset consisting of 10 everyday activities including unknown activities collected from 17 users. The results demonstrate that the activities can be distinguished with the overall accuracies of more than 93% precision and recall for individual evaluation and over 80% precision and recall for subject independent evaluation. These results are really promising for practical applications acquiring the recognition of human activities using mobile devices such as energy expenditure estimation and human behavior monitoring. I. INTRODUCTION Activity recognition is an attractive research field as it has many applications in our lives such as human-computer interaction, healthcare, situated services (i.e. give helpful prompts to the users when they need) [12], energy expenditure estimation [2], human behaviors monitoring [3] etc. Recently, research in activity recognition has achieved significant results. However, most of these activity recognition methods (i.e. data processing and machine learning algorithms) are deployed on desktop or laptop platforms. This approach limits the users performing their activities in pre-setting areas such as a house or a room, and is not relevant for recognizing user’s activities anywhere as mobile devices can do. Moreover, mobile devices are becoming ubiquitous in our lives. A statistical report [4] shows that the number of mobile phones is reaching by 7.3 billion and would exceed world population in 2014. Recent smart phones are integrated by powerful computing resources such as multi-core processor, memory and sensors. Sensors such as accelerometer, thermal, light, direction (for example, magnetic compasses), microphone, vision (i.e. camera), GPS, etc. would not only allow us to be aware of context surroundings, but also can monitor human behaviors. Only a minority of research work in activity recognition is to use mobile phones for deploying the recognition methods (i.e. [5, 6]). Such works often experiment on the dataset containing the small number of activities (i.e. 6 activities [5]) and does not include unknown or null activity class (any activities out of the interested ones) which in practical is important for discriminating activities and rejection, particularly useful for real-time implementation of activity recognition. In this work, we substantially extend and improve our previous pilot work [26] by proposing a new set of features which are effectively invariant to the mobile device’s rotations (i.e. the mobile phones positioned inside the user’s pocket); and deploy two pre-trained statistical machine learning algorithms including Decision Tree C4.5 and hidden Markov models for recognizing human activities in real-time manner using mobile phones. This work is distinct from other works on activity recognitions [7, 8, 9, 10, 11, 12, 13, 14, 15] as we address a set of low-level activities (rather than high-level activity set) and deploy our proposed method on the mobile phones. Compared to other previous works on mobile-phone based activity recognition [5, 6], our activity dataset is more numerous and opened, i.e. included unknown activities. This therefore is significantly more complex than [5, 6]. In addition, we rigorously evaluate our proposed method on an open-dataset (i.e. including unknown activities) collected from 17 subjects who performed 10 different activities without constraints under individual evaluation and subject- independent evaluation protocols. II. RELATED WORK Two common approaches to activity recognition are computer vision based and sensors based. The former analyzes images from video streams of digital cameras that need to be pre-installed in the environments [10, 11] whistle the later analyzes sensor data from different sensors that are incorporated inside the everyday objects or installed in our environments [12, 13, 14, 15, 21], or are worn on different parts of human’s body [7,8,9]. In the first approach, images of the video streams from digital cameras installed in the environments (i.e. [10, 11]) are analyzed. Work by Wu, J. et al. [10] at Intel Research for example, images of the objects being used by the users are pre-processed and then bag-of-word SIFT (scale invariant feature transform) features are then extracted to train statistical Bayesian networks models. The trained models can then be used for recognizing object in use and inferring human activities from observation images. Another work by Duong, T., V. et al. [11] proposed a variation of the hidden Markov models for recognizing home activities in a room using pose features. Such studies obtained good results in the laboratory but they often failed in real-home deployment due to their dependence on the environmental conditions such as light 2015 Seventh International Conference on Knowledge and Systems Engineering 978-1-4673-8013-3/15 $31.00 © 2015 IEEE DOI 10.1109/KSE.2015.43 144