International Journal of Computer Applications (0975 - 8887) Volume 70 - No. 16, May 2013 Vision based Real Time Hand Gesture Recognition Techniques for Human Computer Interaction Archana S. Ghotkar Pune Institute of Computer Technology University of Pune Pune. (INDIA) Gajanan K. Kharate, PhD. Matoshri College Of Engineering and Research Centre University of Pune Nashik. (INDIA) ABSTRACT Natural Interface with Computer using the intelligent approaches is the need of Human Computer Interaction (HCI)applications. In this paper, three techniques were proposed and experimented for interaction with Desktop/Laptop with static hand gesture. All these techniques were using real time approach with different feature descriptors such as Fourier Descriptor(FD),7 Hu moments, Convex Hull and Finger Detection. Real time Recognition efficiency was calculated with respect to recognition time for FD and 7 Hu moments. The 300 samples were trained and stored into database for recognition. For unknown user average recognition time was required 1.7 sec using FD as a feature and 4.6 sec using 7 Hu moments and recognition efficiency was achieved 96% and 98% using 7 Hu and FD respectively. In the second technique New Finger detection algorithm was developed and experimented with Hand tracking system(HTS). In this approach, system was working in dynamic background but gives better result in static background. In the third approach, 3-D Kinect camera was used where hand segmentation was achieved using depth image and finger detection were calculated using Convex Hull. In this approach hand seg- mentation became easier than first two techniques. With all these approaches feature extraction using 7 Hu and FD can be extend for any other HCI application including sign language recognition. Finger counting algorithm can be combined with other descriptor in complex hand sign as a feature. Currently system is working on static hand gestures, further it will be extended to dynamic hand gesture recognition for Indian sign language interpretation. General Terms: Hand Gesture Recognition, Hand segmentation, Hand tracking. Keywords: Human Computer Interface(HCI), Fourier Descriptor(FD), 7 Hu moments, Finger detection, Convex Hull. 1. INTRODUCTION One of the objective of any HCI system is its natural interface. Among other human body part, hand gesture is most natural and powerful communication modality for interaction with computer which needs to be fully explored for HCI. There are two ma- jor approaches for hand gesture recognition: Data Glove, Vi- sion based. Each approach is having its limitation and advan- tages, but vision based approaches are more feasible as com- pared to data glove as user need not to wear cumbersome device like data glove.[1][2] Many researchers are working on Intelli- gent application of HCI such as Intelligent Homes/Offices[3], Intelligent Games[4], Sign language recognition [5] and many more. Thomas and Jaron[6] developed hand-machine interface device that provides real-time gesture, position and orienta- tion information. They used DataGlove containing flex sensors which measure finger blending, positioning and orientation sys- tems and tactile feedback vibrators. William Freeman and Craig Weissman[7] introduced Television control by hand gesture. In their system computer controls the television set through se- rial port commands. The user uses only one gesture: the open hand, facing the camera and he can control television by mov- ing hand. Interpreting human behavior to understand his cultural background is one of the applications of HCI. Matthias et al.[8] introduced this application with major challenges such as grasp- ing culture as a computational term and inferring the user´ s cul- tural background by observable measures. Such an application can be interpreted of the standard language textbook to allow for a deeper understanding of the communication processes that could be achieved by just learning the grammar and words. Af- shin et al.[9] proposed algorithms and application for using hand as an Interface device in virtual and physical spaces. They pro- posed set of applications such as virtual drawing, 3-D model construction and 3-D virtual marble game with hand interface. Christian and Berard[10] described techniques for bare hand in- teraction with computer and tested on application such as con- trolling presentation with hand postures, paint virtually onto the wall. With different applications of HCI, one of the application of Desktop/Laptop interaction with hand interface is chosen for experimentation of algorithms. Natural Interface is preferred to avoid traditional input devices such as keyboard and mouse is the state of the art of HCI application. The same application can be extended for the replacement of mouse by tracking mouse pointer with the tip of the hand finger. Here, five classes of static hand gestures are used for opening most frequently required win- dows applications in real time. In the proposed work three techniques are explored using vision based approach. In the first technique FD and 7 Hu moments were used as a feature descriptor for recognition. In second tech- nique the new Finger Counting algorithm was developed and ex- perimented on HTS [12] system for the same application. Using Convex Hull method and 3D Kinect camera, third technique was explored for the same. The use of Kinect camera made this al- gorithm easy for pre-processing such as subtraction of complex background and skin color detection. The organization of the paper is as follows: Section 2 describes the anticipated data-set used for the system. Methodology is explained in section 3 where three different techniques for hand posture recognition are described in detail with experimental results. Conclusion and Future work is given in section 4. 1