A WOZ Environment for Studying Mutual Adaptive Behaviors in Gesture-based Human-robot Interaction Yong XU, Shinpei TAKEDA and Toyoaki NISHIDA Graduate School of Informatics, Kyoto University Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, Japan {xuyong, takeda}@ii.ist.i.kyoto-u.ac.jp, nishida@i.kyoto-u.ac.jp Abstract Mutual adaptation is considered to be one of the key issues in realizing a human robot interaction (HRI) system. In order to develop a mutually adaptive interactive robot system, the authors adopt a bootstrapping approach comprising three stages: a human-human WOZ (Wizard of OZ) experiment, a human- robot WOZ experiment, and a human-adaptive robot experiment. This paper will focus on the second stage where the authors developed a human-robot WOZ experimental environment and conducted an experiment to evaluate its effectiveness. The primary results suggested the following: the WOZ environment can function well; the operator’s view mode can be considered as a relatively intuitive operational mode for a human WOZ operator; and the realtime video image of the robot’s movement is necessary for the WOZ operator when he/she operates the robot while simultaneously watching the gestural instructions of the instructor. Some adaptive behaviors are observed as well. In addition, the primary results also suggested that the environment can be easily utilized with minor modifications to build a human-adaptive robot experiment. Introduction In recent years, the scope of robotics has extended well beyond industrial applications into several other fields including education, entertainment, and household appliances. In the near future, household robots and service robots are likely to become full-fledged family members. For example, as of May 2006, more than 2 million iRobot Roomba vacuuming robots have been sold worldwide (IROBOT 2007). The capacity of robot-environment- interaction is important for autonomous robots to accomplish simple tasks by interacting with the environment; however, the capacity of human-robot- interaction (HRI) for household or service robots such as nursing-care robots, guide robots, or waiter robots to Copyright © 2007, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. perform cooperative tasks becomes more important. In order to create such robots, aspects such as “easy to use” or “easy to understand” must be emphasized so that non- specialists can easily understand and instruct such robots. Traditional human-computer interfaces may be inappropriate for this purpose since non-specialist users may experience difficulties in interacting with a robot using such interfaces. These interfaces may also cause unnecessary physical and/or mental stress in human users since they must adjust their behavior to the system. In order to realize a powerful interactive human-robot interface, the authors argue that mutual adaptation is a key feature to realize a user-friendly interactive human-robot interface. Mutual adaptation is a phenomenon where two interacting partners adapt to each other during their interaction. In the case of HRI, robots can typically adapt only to some specific people and learn their behaviors. In contrast, the authors attempt to realize a mutually adaptive robot that can not only adapt to any person but also utilize humans’ adaptability to dynamically establish communication protocols. In other words, the same instruction used by different people may contain different meanings, or different reactions are expected from the robot. In particular, when attempting to develop an interactive robot that can gradually adapt to human instructions through nonverbal communication, it is necessary to express the internal states of the robot such that mutual adaptation can occur. This research adopts a bootstrapping approach that comprises three stages: a human-human WOZ (Wizard-of- Oz) experiment, a human-robot WOZ experiment, and a human-adaptive robot experiment. Instead of directly going to the third stage, we first make detailed observations of how people adapt to each other and how people improve the protocols for interacting with robots since it appears to be difficult to design a mutually adaptive robot system directly without any previous knowledge about mutually adaptive behavior. A WOZ experiment is a frequently used method in which participants interact with a system that is assumed to function autonomously. In fact, the system is usually fully or partially operated by a human operator. The goal of such experiments is to observe the behavior of 40