J. Baltes et al. (Eds.): RoboCup 2009, LNAI 5949, pp. 46–57, 2010. © Springer-Verlag Berlin Heidelberg 2010 Real-Time Hand Gesture Recognition for Human Robot Interaction Mauricio Correa 1,2 , Javier Ruiz-del-Solar 1,2 , Rodrigo Verschae 1 , Jong Lee-Ferng 1 , and Nelson Castillo 1 1 Department of Electrical Engineering, Universidad de Chile 2 Center for Mining Technology, Universidad de Chile {jruizd,rverscha,macorrea,jolee}@ing.uchile.cl Abstract. In this article a hand gesture recognition system that allows interact- ing with a service robot, in dynamic environments and in real-time, is proposed. The system detects hands and static gestures using cascade of boosted classifi- ers, and recognize dynamic gestures by computing temporal statistics of the hand’s positions and velocities, and classifying these features using a Bayes classifier. The main novelty of the proposed approach is the use of context in- formation to adapt continuously the skin model used in the detection of hand candidates, to restrict the image’s regions that need to be analyzed, and to cut down the number of scales that need to be considered in the hand-searching and gesture-recognition processes. The system performance is validated in real video sequences. In average the system recognized static gestures in 70% of the cases, dynamic gestures in 75% of them, and it runs at a variable speed of 5- 10 frames per second. Keywords: dynamic hand gesture recognition, static hand gesture recognition, context, human robot interaction, RoboCup @Home. 1 Introduction Hand gestures are extensively employed in human non-verbal communication. They allow to express orders (e.g. “stop”), mood state (e.g. “victory” gesture), or to trans- mit some basic cardinal information (e.g. “two”). In addition, in some special situa- tions they can be the only way of communicating, as in the cases of deaf people (sign language) and police’s traffic coordination in the absence of traffic lights. Thus, it seems convenient that human-robot interfaces incorporate hand gesture recognition capabilities. For instance, we would like to have the possibility of trans- mitting simple orders to personal robots using hand gestures. The recognition of hand gestures requires both hand’s detection and gesture’s recognition. Both tasks are very challenging, mainly due to the variability of the possible hand gestures (signs), and because hands are complex, deformable objects (a hand has more than 25 degrees of freedom, considering fingers, wrist and elbow joints) that are very difficult to detect in dynamic environments with cluttered backgrounds and variable illumination. Several hand detection and hand gesture recognition systems have been proposed. Early systems usually require markers or colored gloves to make the recognition eas- ier. Second generation methods use low-level features as color (skin detection) [4][5],