Proceedings of the 1999 IEEE International Workshop on Robot and Human Interaction Pisa, Italy - September 1999 Gesture Recognition for Human-Friendly Interface in Designer - Consumer Cooperate Design Systi;m Hideyuki SAWADA*, Seiji UTA* and Shuji HASHIMOTO* * ( sawada, hata} @eng.kagawa-u.ac.jp shuji@shalab.phys.waseda.ac:.jp * Dept. of Intelligent Mechanical Systems Engineering, Faculty of Engineering, KAGA,WA University 1-1 , Saiwai-choy Takamatsu-city, Kagawa, 760-8526, Japan **Dept. of Applied Physics, School of Science and Engineering, WASEDA University 3-4-1, Okubo, Shinjuku-ku, Tokyo, 169-8555, Japan Abstract A future system for flexible manufacturing must be a sensor-complex for multi-modality with high intelligence, since such system should work with human who has a variery of expression ways and acceptance channels. This paper describes three topics from our recent researches on gesture and haptic interfnces which can be introduced for human-friendly integace in designer-consumer cooperative production system. The first one is a gesture information acquisition method by using image processing technique. The second one is another gesture recognition system by employing accelerometers to measure the forces applied to the body. A new haptic-communication device is introduced as the third topic of the paper: It is able not only to detect grasping force and the device orientation, but also to display the tactile sensation using the vibration motors to realize human-machine and human-human haptic communications. 1. Introduction As human has a variety of expression ways and acceptance channels, the technology for human communication must have multi-modality. A system to work with human has to understand not only the user's intention but also the environmental condition with multi- modal sensing ability. Thus, the future system for flexible manufacturing must be a sensor-complex with high intelligence. The different sensory information will be integrated organically and the system actively works with human using various communication channels such as sound, voice, facial expressions, and motion of the head, eyes, arms and legs. Such system will try to understand the will of a human user while iterating bi-directional communication. These systems may be a sort of human-like environment or agent to perform smoothly a cooperative task together with human. Among a variety of human communication media, gesture and touch feeling play important roles in our daily life as nonverbal media. Therefore, it is important and effective to introduce haptic and gesture communication into production processes such as display, design and manufacturing[ 11. This paper describes three topics from our recent researches on gesture ancl haptic interfaces. The first one is a gesture information acquisition system for display by using image ]processing technique to be used in a virtual shopping[2]. A user can point to goods while walking through the virtual shops. The second one is another gesture recognition system[3][4]. Although most of the reported works to introduce the body movement into human-machine interface treat the shape or the position of body, we developed a new type of gesture recognition system for design by employing accelerometers to measure the forces applied to the bcdy. A new communication device named GraspCom is introduced as the third topic of the paper[5]. GraspCom can detect the grasping force and the orientation by using a pressure sensor and gravitation sensor. At the same time, it can displ'ay the tactile sensation using the vibration motors to realize human-machine and human-human haptic communicatioi~s in manufacturing. All of the proposed human gesture handling system can be introduced for human friendly interface in designer- consumer cooperative production system. 2. Two Approaches to Computerized Gesture Recognition For the computerized gesture acquisition, a gesture has to be measured by some sensors first, then the sensor data are processed in the computer. Measurement of human gestures has been studied by mainly two approaches: one is to use image processing techniques, and the other is to employ wearable sensing devices. The former techniques are actively applied to the realtime motion analysis by restricting the measurement condition. Image processing technique lets the performer be free from wearing devices and sensors, an universal algorithm for the realtime motion analysis is, however, 0-7803-5841 -4/99/$10.000 1999 IEEE 400