1949-3045 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2016.2569098, IEEE Transactions on Affective Computing Robust Facial Expression Recognition for MuCI: A Comprehensive Neuromuscular Signal Analysis Mahyar Hamedi, Member, EMBS, Sh-Hussain Salleh, Member, IEEE, Chee-Ming Ting, Member, IEEE, Mehdi Astaraki and Alias Mohd Noor Abstract—This paper presents a comprehensive study on the analysis of neuromuscular signal activities to recognize eleven facial expressions for Muscle Computer Interfacing applications. A robust denoising protocol comprised of Wavelet transform and Kalman filtering is proposed to enhance the electromyogram (EMG) signal-to-noise ratio and improve classification performance. The effectiveness of eight different time-domain facial EMG features on system performance is examined and compared in order to identify the most discriminative one. Fourteen pattern recognition-based algorithms are employed to classify the extracted features. These classifiers are evaluated in terms of classification accuracy and processing time. Finally, the best methods that obtain almost identical system performance are compared through the Normalized Mutual Information (NMI) criterion and a repeated measure analysis of variance (ANOVA) for a statistical significant test.To clarify the impact of signal denoising, all considered EMG features and classifiers are assessed with and without this stage. Results show that: (1) the proposed denosing step significantly improves the system performance; (2) Root Mean Square is the most discriminative facial EMG feature; (3) discriminant analysis when the parameters are estimated by the Maximum Likelihood algorithm achieves the highest classification accuracy and NMI; however, ANOVA reveals no significant difference among the best methods with almost similar performance. Index Terms—Facial Neuromuscular Activity, Muscle Computer Interaction (MuCI), EMG Denoising, Feature Extraction, Classification, Facial Expression Recognition. —————————— —————————— 1 INTRODUCTION MPROVING life quality has always been one of the priorities amongst scientists, physicians and research- ers. In recent years, incorporation of electron- ic/computer/mechanics with biology/medicine study fields has led to numerous invaluable achievements. One of the most promising results to assist the locked-in pa- tients with crucial disabilities, amputees and the elderly is Human Computer Interaction (HCI) technology which is an approach to transmit the information between humans and a computer [1]. Developing prosthesis limbs, robotic arms and controlling the assistive devices like wheel- chairs are counted as important applications in this area. Since the reliability and flexibility of such systems are essential for the users, various techniques and types of interfaces have been suggested and employed. Recogniz- ing the user’s body movements like those of the head, hand and wrist through bioelectrical activities and con- verting them into computer control commands have been focused recently. However, crucially disabled people cannot even move their neck, and the only existing way of communication for them is through facial expressions neuromuscular activities (electromyograms (EMGs)) or brain waves (electroencephalograms (EEGs)). Such HCIs are called Muscle Computer Interaction (MuCI) and Brain Computer Interaction (BCI). It is reported that BCI is only preferred when the use of MuCIs is not feasible [2]. There have been numerous studies on the potential of MuCI systems, including multifunction prosthesis [3-6], power exoskeleton control [7], wheelchairs [8-9], robotic control [10] and grasping control [11]. For affective communication between the user and computer, researchers much consider how facial expres- sions can be recognized efficiently during MuCI. In order to establish a robust MuCI system based on facial EMGs, several essential factors have to be taken into account, particularly the recording protocol and analysis steps. Since the comfort and friendliness of these systems are vital for users, non-invasive surface electromyography (SEMG) should be considered as the most convenient way to record the underlying activities of muscle contrac- tion. SEMG characteristics, such as amplitude and energy, differ from 0 to 10 mV and 0 to 500 Hz respectively based on the activated muscle and ratio of contraction. These signals are inherently stochastic, non-stationary and noisy, contaminated by a variety of internal (e.g. EEGs, electrooculogram (EOGs)) and external (e.g. recording equipment, ambient noise, motion artifacts) factors [12]. The success of MuCIs depends substantially on classifica- tion performance and computational cost consumed dur- ing processing to provide a reliable trade-off between accuracy and speed. The significant factors in pattern I ———————————————— M. H., Sh-H. S., C-M. T., and A. M. N. are with Center for Biomedical Engineering, Universiti Teknologi Malaysia, 81310, Johor Bahru, Malay- sia. E-mails: hamedi.mahyar@ieee.org, hussain@fke.utm.my, cmting@utm.my, alias@mail.fkm.utm.my M. A. is at department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran. E-mail: astara- kee@yahoo.ca.