A. Camurri and G. Volpe (Eds.): GW 2003, LNAI 2915, pp. 386–398, 2004. © Springer-Verlag Berlin Heidelberg 2004 Intuitive Manipulation of a Haptic Monitor for the Gestural Human-Computer Interaction Hidefumi Moritani, Yuki Kawai, and Hideyuki Sawada Faculty of Engineering, Kagawa University 2217-20, Hayashi-cho, Takamatsu-city, Kagawa, 761-0396, Japan sawada@eng.kagawa-u.ac.jp http://www.eng.kagawa-u.ac.jp/~sawada/index_e.html Abstract. A display device called intuition-driven monitor will be introduced. The device has been developed by combining a LCD, a flexible arm suspending the display, angle sensors and a CCD camera, and is directly manipulated by a user with his direct operations by hands, eye-gaze and facial expressions. A user is able to explore virtual space and manipulate virtual environment with gestures and the sense of haptics. 1 Introduction Humans are able to communicate with each other by using not only verbal media but also the five senses effectively using their body parts. Information transmitted through non-verbal media directly affects our emotions and feelings, and especially gesture plays an important role for the emotional human communication. With the wide- spread availability of computers which have been used for the calculation and infor- mation processing, they are now used as an extensive tool for the communication and creative activities. The development of Graphical User Interface (GUI) has made the computer familiar to the general public, but the user’s inputs to the computer are mainly conducted by using a mouse and a keyboard, which is far from the way we use in the human to human communication. If a computer is able to understand what the human requests with his non-verbal means, and also to present an answer appealing to human mind and emotions, a new communication system will be realized. We are surrounded by various computer graphics (CGs) and images generated by a computer with the development of the image processing technology, however, there still exists bothersome manipulations in handing them especially for beginners and elderly people, since the operation to the virtual object is indirect and the operation itself is difficult and complicated. This paper introduces a new human-interface de- vice, which can be used for the direct manipulations of a CG and Virtual Reality (VR) space. A display device called intuition-driven monitor has been developed by combining a liquid crystal display (LCD), a flexible arm, angle sensors and a CCD camera, and an application system realizing a gestural human-computer interaction is proposed in this paper. A user is able to look a virtual 3D object into the display from an arbitrary direction by directly manipulating the device at his will. The system also accepts user’s intuitive operations of the monitor based on the recognition algorithms of mo- tion and facial expressions, with which the user is able to transmit his intention to the