An Iconic Gesture is Worth More Than A Thousand Words Timothy Marsh Human Computer Interaction Group, Department of Computer Science, University of York, YO10 5DD, UK. tmarsh@.cs.york.ac.uk Abstract In our minds, we can create an imaginary environment of shapes and objects. A natural and intuitive way to externalize these imaginary shapes is through our hands by simply sketching or tracing its outline in space. If we could capture and then interpret our handshapes, then we will be able to express spatial information in computer generated graphical environments in a simple and extremely powerful way. This paper describes the on-going research to employ iconic gestural-based human-computer interaction; a novel interaction technique for the input of spatial information. Introduction Using input interactive devices, such as, a 2D mouse, keyboard, trackball, puck and tablet, 3D mouse, 3D stylus, or spaceball, to express spatial information in computer generated graphical environments, such as, computer aided design, virtual reality, or modeling systems can be quite restrictive. This is because, our expressive capabilities are confined to the method of operation imposed on us by the input interactive device. One way to augment the transfer of spatial information, is to increase the expressive capabilities of the user. A very natural and intuitive way to express spatial information, is simply to use the human hand. The human hand is an extremely complex structure having 27 degrees of freedom (DOF) that allow us to articulate a near infinite number of handshapes and movements performed in an almost graceful and choreographed manner. If we can capture and interpret handshapes, then we will be able to express spatial information in computer generated graphical environments in a simple and extremely powerful way. Human-Human Hand Gestures There are two main human-human hand gestural communication systems in everyday use. Firstly, sign language, such as, British Sign Language (BSL) [2] and American Sign Language (ASL) [5] et al., and secondly, natural gestures that are used to accompany speech [10]. There are many taxonomies and classifications of hand shapes and movements. Generally, all handshapes or gestures fall under one of 4 categories [10]: • Arbitrary: no obvious relationship between a gesture and its meaning • Indexic: pointing (deictic) or indexing a person, object or location • Iconic: drawing or tracing an outline of a picture of a shape or object in space or the hand(s) represent the shape or object itself [4] • Pantomimic: mime or act out the meaning Research from both sign language [4] and nature gestures [10] has shown that the articulation of shapes and objects is performed using iconic gestures. This was also shown to be true in a recent study conducted by the author [6]. The study was carried out to test whether iconic hand gestures could be used to articulate spatial information within computer generated graphical environments. The results showed that subjects preferred to articulate 2D and 3D shapes, that are the basic building blocks found in most computer graphics modeling systems, and everyday 3D objects, using iconic gestures [6]. In particular, subjects preferred to articulate 2D and 3D shapes and objects using virtual depiction, that is, drawing or tracing the outline of a picture or shape in space [4]. Similar studies have shown that iconic gestures are used when subjects wish to externalize pictorial information [1].