Abstract—Drowsiness is one of the main causes of severe traffic accidents occurring in our daily life. In order to reduce the number of drowsiness-induced accidents, various researches have been conducted with the aim of finding practical and non-invasive drowsiness detection systems by using behavioral measuring techniques. Many of the previous works on behavioral measuring techniques have mainly focused on the analysis of eye closure and blinking of the driver. It is recently that more attention started to shift to inclusion of other facial expressions and only few, among those researches, have been done on the analysis of temporal dynamics of facial expressions for drowsiness detection. In this paper we propose a new method of analyzing the facial expression of the driver through Hidden Markov Model (HMM) based dynamic modeling to detect drowsiness. We have implemented the algorithm using a simulated driving setup. Experimental results verified the effectiveness of the proposed method. Keywords Drowsiness detection, facial expression, SVM, HMM I. INTRODUCTION According to the US National Highway Traffic Safety Administration, approximately 100,000 crashes occur in US each year due to drivers’ drowsiness [1]. In an effort to prevent such crashes, the U.S. Department of Transportation has taken a notable initiative in the making of intelligent vehicles. In this context, the development of robust and practical drowsiness detection system is a crucial step. Many researches are being undertaken to develop better ways of detecting drowsiness, such as the behavioral, physiological changes of the driver, the steering wheel movement or vehicle responses, etc. It is critical that a drowsiness detection system should be accurate and reliable when they are deployed for commercial use. Even if vehicle based drowsiness detection systems are noninvasive, they have been found to be very unreliable as they depend on the nature of the road, the vehicle, the traffic, the way the driver drives and other external factors. Behavioral measuring methods are more reliable than vehicle based systems and are also noninvasive and easier to be implemented. However, many of the commercially available behavioral measuring methods mainly focus on eye closure and not on other facial expressions. In this paper, we propose a drowsiness detection method that includes other facial motions and behavioral changes in This work is partially supported by the NSF grant CISE/IIS 1231671 and National Natural Science Foundation of China under Grants 61328302 and 61222310. Eyosiyas Tadesse and Weihua Sheng (contacting author) are with the School of Electrical and Computer Engineering, Oklahoma State University, Stillwater,OK74078, USA (e-mail: weihua.sheng@okstate.edu). Meiqin Liu is with the College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China. addition to eye closure. We also adopted a dynamic model for analyzing the facial expressions to determine drowsiness which will significantly improve the reliability of drowsiness detection. We first developed a frame based drowsiness detection algorithm. Then we introduced drowsiness detection based on temporal analysis of facial expression and demonstrated its advantage over frame based drowsiness detection through experiments. We have optimized the system parameters to maximize the accuracy and speed of detection. We conducted the experiments in a simulated driving environment. II. RELATED WORK Real time drowsiness detection has been implemented using different detection techniques analyzing various types of input data. The first approach is analyzing the measurement of physiological activities of the human body, such as brain wave (EEG), heart rate or pulse rate [2]. Even though the measurements and their correlation with the alertness of the driver is quite accurate, they are not practical as it would require the driver to always wear the sensing devices and the hardware cost is too high for commercial use. The second approach makes use of vehicle based measuring techniques to detect the drowsiness of the driver. In this approach, the driver’s drowsiness is measured by analyzing the different controller signals of the vehicle, such as steering wheel movement, pressure from the gas and brake pedal, speed of the vehicle, change in shift lever and deviation from lane position [3]. The measurements of these signals are obtained from sensors equipped in the vehicle. Among the vehicle based metrics that have been used to determine drowsiness, steering wheel movement has been shown to give better detection capability [4]. The steering angle is constantly measured by a sensor and the change in angle movement is checked if it is within or exceeds a specified threshold. Even though vehicle based approaches are noninvasive, they are not reliable in detecting drowsiness as their performance is highly affected by the nature of the road, the way the driver drives, the traffic or a driving impediment other than being drowsy. The third approach is behavioral measuring that makes use of computer vision techniques to detect the changes in driver’s facial expressions [5]. Existing works in this area have mainly relied on analyzing the percentage of closure (PERCLOS) of the driver’s eyes. The first step in such systems is face and eye detection. Li et al. [6] performed successive image filtering techniques such as image subtraction, morphologically closed operations and binarization, and finally counted the number of pixels around the eyes region to detect eye closure. Liu et al. [7] extracted simple features from the temporal difference image and used Driver Drowsiness Detection through HMM based Dynamic Modeling Eyosiyas Tadesse, Weihua Sheng, Meiqin Liu 2014 IEEE International Conference on Robotics & Automation (ICRA) Hong Kong Convention and Exhibition Center May 31 - June 7, 2014. Hong Kong, China 978-1-4799-3685-4/14/$31.00 ©2014 IEEE 4003