Proceedings of the 4th International Conference on Autonomous Robots and Agents, Feb 10-12, 2009, Wellington, New Zealand Viewing Comics on Robots Hang-Bong Kang Dept. of Media Eng. Catholic University of Korea #43-1 Yokkok 2-dong Wonmi-Gu Buchan City, Gyonggi-Do, Korea hbkang@catholic.ac.kr Abstract- For efficient viewing of the comics, it is desirable for the user to non-linearly control the viewing order of panels in comics and manipulate size and colors of objects in the specified panel. In this paper, we present human robot interaction techniques using face pose, facial expression and hand gesture for viewing comics on the robot. The user's face pose is used to control viewing order of panels and Hand gesture is used to manipulate the objects or robot control. Facial expression such as happiness, sadness, fear and anger is used to change the colors of foreground and background objects in comics. Our proposed method shows desirable results in viewing comics on the entertainment robots. Keywords- Human robot interaction; face pose; facial expression; gesture recognition Myung-Ho Ju Dept. of Computer Eng. Catholic University of Korea #43-1 Yokkok 2-dong Wonmi-Gu Buchon City, Gyonggi-Do, Korea hangeI5@catholic.ac.kr one panel using her hand gesture and manipulate the object in the panel. Fig. 2 shows that the activated object is translated or rotated according to the user's hand gesture. Based on the contents in comics, the robot shows some responses such as "move forward", "move backward" or "make a circle". As a result, a user can appreciate the comics in his own style. Currently, various types of robots such as intelligent service robots, entertainment robots etc., are developed. One of the key issues in those robots is the human-robot interaction (HRI). For natural HRI, it is desirable for a robot to recognize and interact with the user's face pose, gesture, voice, and facial expression. For example, entertainment robots are new media with mobility to transfer various contents to audiences. Children can read and hear fairy tales, comics and sing a song through the robots. Fig. 1 shows human robot interaction using face pose, facial expression and hand gesture. Conventional viewing method through the robots is usually linear and limited. For efficient viewing of the comics, in particular, it is desirable for the user to non-linearly control the viewing order of panels in comics and manipulate size and colors of objects in the specified panel. To do this, the comic contents should be prepared with multi-layer structure. In other words, the panel of comics is generated from foreground layer and background layer. Several objects are segmented in the foreground layer. As HRI techniques for viewing contents like comics, we use face pose, facial expression and hand gesture recognition. Using these techniques, a user can control the viewing order of panels in comics and manipulate object's shape, position and color. By using HRI techniques, contents in comics can be displayed in non-linear order and objects in the panel can be modified to different color, size, orientation or location from original contents. For example, a user views each panel in comics using her face pose like Fig. 2. Then he can zoom in I. INTRODUCTION Figure 1. Face pose, facial expression and hand gesture based human robot interaction. Figure 2. The viewing panel is changed using face pose. Objects are manipulated by hand gesture recognition. 978-1-4244-2713-0/09/$25.00 ©2009 IEEE 480