Hand Feature Detection from Skin Color Model with Complex Background Abstract- We present a new technique for extraction of hand region from complex background and consequently detection of the fingertips, which we call hand features, from color images. Our construction is primarily based on an adaptive color model generation for hand segmentation followed by smoothing algorithm. We present a thinning algorithm followed by the construction of convex envelope to detect possible points for the fingertips. We demonstrate that the correct points for the fingertips can be selected heuristically through interpoints distance calculation. Finally, we show the effectiveness of the proposed method by experimenting with images of different background complexity and have achieved very promising results. Keywords: fingertips detection, hand tracking, smoothing I. INTRODUCTION The detection of fingertips can be considered as an important step for certain applications which include gesture recognitions, sign language understanding and other Human-Computer Interaction (HCI) areas. One of the good fingertips detection algorithms developed can be found in [1], [5] in which the fingers’ locations are found through simple skin detection. There are several reasons which make hand tracking difficult to achieve [2], [6]. The first reason is due to the fact that the hands are deformable and articulated object which is hard to be modeled accurately. Secondly, in many applications such as surveillance, hands are usually tracked in an uncontrolled environment e.g. varying lighting condition and cluttered scenes. Thirdly, for applications that need to track both hands simultaneously, the problem of one hand occluding the other need to be taken into consideration. Finally, certain type of clothing may occlude the hands whether partial or full and this may affect the accuracy of the detection and tracking. Using color information to detect hands is a good strategy in order to simplify the task of fingertips localization in complex environments. Among many existing color models, YCbCr has been proven to be able to detect skin region of the hands effectively because it has better clustering capabilities than other color models. RGB color model which is based on human perception of colors is unsuitable to be used as it is very sensitive to illumination changes and lighting conditions. Even though YCbCr can be used as an acceptable color model to detect skin region, it has some drawbacks in which its sensitivity to the light source is still high and the detection will become worse for complex background case. One way to overcome this is by taking out the luminance component of the YCbCr. This will increase the robustness of the skin detection in the case where variations in lighting conditions cannot be avoided [3]. The advantage of using skin color for detecting hands and localize fingertips is that it is invariant to scale and orientation. In [4], color cue is used to segment the hand from the background. Once the hands are detected, a counter vector which contains the edges of the detected hands is created. The counter vector will then be processed to obtain the location of the fingertips. This is achieved by segmenting the palm from the hand using morphological opening operation with elliptical structuring element to find the fingers. Another approach is to articulate object detection applied to the problem of fingertips. The system is able to detect a hand in cluttered images with different relative positions of a finger with respect to the position of the whole hand [5]. The techniques of this method to utilized hand detection approach, for a possible match between the image and an affine-transformed representation of the finger’s appearance and shape model.also represent in a 2D reference frame and projected into the image to prove the presence of skin and motion pixels and remove any noisy edge pixels. [5]. In [6], the detection of fingertips was done using markers which are placed on the fingertips. The detected fingertips were used to recognize chords played on a guitar. This process was done in real time by recognizing the patterns of fingers pushing positions detected from input images. By employing triangulation method on stereo cameras, the 3D positions of fingertips were recognized when a guitar string was pressed. To segment the hand, colored skin objects are detected by utilizing Bayesian classifier that is bootstrapped with a small set of training data and refined through an off- line iterative training procedure [6]. We propose an efficient method to detect the fingertips in color images. We employ an adaptive skin color model for segmenting hands from the background. Fingertips are then detected using a modified smoothing and thinning algorithm followed by interpoints distance measurement. We organize Ahmad Yahya Dawod Faculty of Information Technology, Multimedia University, Cyberjaya, Malaysia ahmad.yahya.dawod07@mmu.edu.my Junaidi Abdullah Faculty of Information Technology, Multimedia University, Cyberjaya, Malaysia junaidi@mmu.edu.my Md. Jahangir Alam Faculty of Information Technology, Multimedia University, Cyberjaya, Malaysia md.jahangir.alam@mmu.edu.my Annual International Conference on Advanced Topics in Artificial Intelligence (ATAI 2010) Copyright © GSTF 2010 ISBN: 978-981-08-7656-2 doi:10.5176/978-981-08-7656-2ATAI2010-29 A-23