Development and User Testing of the Gestural Joystick for Gloves-On Hazardous Environments Jaewook Bae 1 , Amy Larson 2 , Richard M. Voyles 3 , Roy Godzdanker 4 , Jan Pearce 5 1,2 EE, CS Department, University of Minnesota, MN, 55455, USA, e-mail:(baeja, larson)@cs.umn.edu 3,4 CE Department, University of Denver, Co, 80208, USA, e-mail:(rvoyles, roy.godzdanker)@du.edu 5 Math and CS Department, Berea College, KY, 40404, USA, e-mail:pearcej@berea.edu Abstract— For controlling robots in an urban search and rescue (USAR) application, we present a wearable joystick with improved sensing capability as well as Giant Magneto- Resistance(GMR) sensor model for rare-earth magnet. Scientists have already begun to try to apply existing interactive devices to control USAR robots in a disaster. In a USAR task, the selection out of numerous interactive devices has to be carefully concerned. Some clumsy or irritating interactive devices can result in the burden of the carrying. We present a wearable joystick based on unencumbered mechanism. The features of the wearable joystick include easy and wire-free installation into regular gloves. We improved the hardware structure for sensor pad and alignment of magnets and applied band-typed sensor pad to completely wrap up wrist. This band-typed mechanism allows us to reliably obtain sensor data. However, in order to determine the performance of the new device, we perceived that we required an adequate evaluation method. We adopt movement time and Fractal dimension evaluation method to describe the degree of control path tortuosity. In addition, we present experimental results in both computer screen and real USAR robot test. I. I NTRODUCTION Robotic technology for urban search and rescue (USAR) is an active area of research and development. Over the last decade, the research deployment of USAR robots has become more common during training exercises and even during some actual emergencies. For instance, after the September 11th attack on the World Trade Center, robot scientists contributed to USAR teams to search for trapped victims[1]. USAR robots are currently controlled exclusively via teleoperation but traditional input devices may be inappropriate [2]. Traditional human/computer interaction is hampered by mandated protec- tive gear including safety glasses, hard hats, respirators, and, most importantly, gloves. Gloves have been found to reduce operator effectiveness by [3]. Since operators in USAR are required to wear heavy gloves to insulate themselves from the hazardous environment, we began to exploit these gloves as an opportunity to embed wearable sensors [4]. In prior work, we presented a prototype wearable input interface for a specific type of USAR application which we implemented with the USAR robot, TerminatorBot [4]. The TerminatorBot has been developed for use in core-bored inspection during USAR tasks. The idea of the wearable input interface began with the burden of the input device or a controller for the TerminatorBot. Conventional input devices, such as mouse, keyboard and joystick, are still cumbersome to the operators wearing safety gears, even though they are small enough to carry. Conventional glove interfaces, such as Power Glove, CyberGlove, MIT LED glove and etc [5], have shown to be highly accurate for the recognition of various hand gestures [6], [7], [8], [9]. However, these glove interfaces require complex and fragile sensor structure and wires and are not suitable for use in rugged environments. We are developing a gestural joystick for USAR tasks. This work presents the development of a wearable joystick that focuses on unencumbered installation and operation based on an intuitive paradigm. We not only tackle the problem from a functional standpoint, but also exploit a novel metric for usability and ergonomic evaluation. Previous work in human-robot interaction (HRI) devices in USAR concerned only about robotics experts as operators. The technical knowledge gap between a general rescuer and a robotics expert frequently became an obstacle to obtain an ultimate goal of user interactive devices. From this lesson, some scientists[1], [10] have begun to understand design parameters not previously considered. Recently, the scientists have focused on finding design parameters of interrelationship, collaboration, video feedback, and dialogue in human-robot systems[1], [10]. In the evaluation, metrics are necessary to measure how much a rescuer can cooperate with USAR robots under a certain task[10], [11]. The metrics for USAR robots are still in its infancy. Steinfeld[11] tried to generalize common metrics for task-oriented HRI and built a standardized framework in HRI. However, the result of the framework was not enough to be applied to such a specific USAR domain and also still in progress. In this literature, we improve the sensing accuracy and magnetic sensor structure of the system and evaluate the system with two metrics, operational time and path tortuosity. Since the evaluation of an input device is strongly based on the usability test, the performance of the wearable joystick relies on the degree of user satisfaction and user efficacy. The usability tests are also important at the design process as well as at the evaluation of an input device as results can affect the design parameters. II. GESTURAL J OYSTICK The gestural joystick provides unencumbered wearability that can be equipped on many styles of traditional gloves. Moreover, the hardware structure of the system can be quite simple and robust in terms of operation and installation. The embedded rare-earth magnets are relatively small and the sensor pad is worn on the wrist or embedded in the