VISUAL RECOGNITION OF HAND POSTURES FOR INTERACTING WITH VIRTUAL ENVIRONMENTS Radu Daniel VATAVU Ştefan-Gheorghe PENTIUC "Stefan cel Mare" University of Suceava str.Universitatii nr.13, RO-720229 Suceava raduvro@yahoo.com, pentiuc@eed.usv.ro Christophe CHAILLOU Laurent GRISONI Samuel DEGRANDE Laboratoire d’Informatique Fondamentale de Lille Universite des Sciences et Technologies de Lille Christophe.Chaillou@lifl.fr, Laurent.Grisoni@lifl.fr, Samuel.Degrande@lifl.fr Abstract. The paper addresses the problem of visual recognition of several hand postures corresponding to a few operations commonly performed in virtual environments, such as: object selection, translation, rotation and resizing. Processing is performed in a top-view scenario with a top-mounted camera that monitors the user’s hands on the working desktop. By careful choosing and controlling of the scene and lighting conditions, hands segmentation is fast and robust which increases the performances of the hand posture classifier. The chosen classifier was a multilayered perceptron with three layers. By keeping all the processing at a low level of complexity and by considering an appropriate control of the environment, we obtain a real time 25 fps functional system with high detection and recognition accuracy results. Keywords: computer vision, hands detection, human computer interaction, multilayered perceptron. Introduction Human gestures are perceived as a natural mean for interacting and conveying information [1] and gesture based interfaces are looked upon as ideal with respect to the human computer interaction techniques [2, 3]. Even more, video based gesture recognition has the main attraction of not being intrusive and of not requiring the user to wear additional equipments or devices, giving in the end a comfortable feeling of naturalness. For the special case of virtual environments, appropriate human computer interfaces are in order. VR appears as an impoverished version of the physical world with incomplete sensory cues and simplified and inconsistent world models. The virtual experience is influenced by experiential, cognitive, perceptual and motor differences between users. Hence, the interaction technology should be appropriate so that the overall user experience in the virtual environment should not be diminished. The paper discusses the visual recognition of a set of hand postures that have been selected in accordance to several commonly commands that may be performed for interacting with virtual objects inside VR environments. Hand postures have been identified for common operations such as: selection, translation, rotation and resize of virtual objects. Posture recognition is carried out using a multilayered perceptron with a three layers structure. Previous work Basic research has been conducted under the general term of gesture recognition, most of which is centered on hand recognition. Gesture recognition can be grouped in two major categories: gesture acquisition (using techniques specific to video and image processing) and actual gesture recognition (techniques that are specific to pattern recognition). Gesture acquisition considers the detection and tracking of an object of interest (for example the hand