International Journal of Knowledge-based and Intelligent Engineering Systems 14 (2010) 139–152 139 DOI 10.3233/KES-2010-0197 IOS Press Recognition of dynamic gestures in arabic sign language using two stages hierarchical scheme Mohammed Al-Rousan * , Omar Al-Jarrah and Mohammed Al-Hammouri Department of Computer Engineering, Jordan University of Science and Technology, Irbid, Jordan Abstract. Most of the existing work on Arabic sign language (ArSL) recognition focuses on static gestures, while there is a growing need for recognition of continuous gestures. In this work, we develop a system that makes automatic translation of dynamic gestures in the Arabic Sign Language (ArSL) using two stages (Hierarchical) scheme. The system is composed of two stages: the first stage recognizes the group of the gesturer and the second stage recognizes the gestures within the groups. Spatial domain analysis is used for features extraction from the hands and face regions, which are classified using Hidden Markov Model (HMM). The extracted features include eccentricity of the hand region, coordinate of the center of the hand region, direction angle of the hand region, and the hand vector that represents the shape of the hand. These features are scale and translation invariant. We have used two types of features: simple and complex. The simple features comprise six features and the complex comprises 17 features. The complex features include 11 hand vectors which are not included in the simple features. The recognition rate for the signer-dependent is 92.5% and for the signer-independent is 70.5%. Keywords: Sign language recognition, arabic sign language, hidden markov model, spatial domain analysis, dynamic gestures 1. Introduction Sign language is the basic method of communication between deaf people. However, one might need to communicate with a deaf person. In this case, an expert translator signer is needed to translate spoken language to the corresponding sign language and vice versa. This makes the communication process with deaf people more difficult since a signer is needed every time. Most of previous studies on sign languages are based on vision or glove-based methods [1]. In the vision based method, the system uses image processing tech- niques to recognize the gestures without imposing any limitation on the user. While in glove-based method, the user needs to wear special devices, like gloves or markers, to provide the system with data related to the hand shape and motion. * Corresponding author. E-mail: alrousan@just.edu.jo. There are several challenges in the continuous sign language recognition process. First, the signs may vary in speed and position even if the gesture is performed twice by the same signer. Second, the gestures may overlap which makes it difficult to detect their bound- aries. Finally, a sign may be affected by the preced- ing signs and may have an effect on the subsequent signs, which is known as the co-articulation problem; the starting position for a sign depends on the ending position of the previous sign and the ending position for a sign affects the starting position of the next sign. In other words, the co-articulation means the extra move- ments between two consecutive gestures. Many researchers have been working on hand ges- tures in many sign languages. Unlike other sign lan- guages such as the American Sign Language (ASL), the Chinese Sign Language (CSL), and the Australian Sign Language (Auslan) [2–5], the Arabic Sign Language (ArSL) has not received a lot of attention. Chen et al. [1] developed a vision-based hand ges- ture recognition system for Taiwanese Sign Language ISSN 1327-2314/10/$27.50  2010 – IOS Press and the authors. All rights reserved