Abstract—An intuitive user interface for the teleoperation of mobile rescue robots is one key feature for a successful exploration of inaccessible and no-go areas. Therefore, we have developed a novel framework to embed a flexible and modular user interface into a complete 3-D virtual reality simulation system. Our approach is based on a client-server architecture to allow for a collaborative control of the rescue robot together with multiple clients on demand. Further, it is important that the user interface is not restricted to any specific type of mobile robot. Therefore, our flexible approach allows for the operation of different robot types with a consistent concept and user interface. In laboratory tests, we have evaluated the validity and effectiveness of our approach with the help of two different robot platforms and several input devices. As a result, an untrained person can intuitively teleoperate both robots without needing a familiarization time when changing the operating robot. Keywords— teleoperation of mobile robots, augmented reality, user interface, virtual reality. I. INTRODUCTION HE enormous improvements in miniaturization and processing power now allow robots to cover new and wider areas of activity. In particular, applications are emerging for mobile robots in rescue operations. For example, these robots can collect information about victims or about building structures in no-go areas that are inaccessible for rescue workers. In this field of research, different approaches exist for the design and control of rescue robots. Currently, most approaches try to develop autonomous victim-finding and map-generating robots. In contrast to this, our interests are not autonomous robots. Instead, we will focus on the teleoperating approach because we want to incorporate the experience and intuition of a human teleoperator. We are convinced that the advantages of continuous teleoperation of mobile robots in many scenarios cannot be fully exploited yet because of inadequate user Juergen Rossmann heads the Institute of Man-Machine Interaction, RWTH Aachen University, 52074 Aachen, Germany (e-mail: rossmann@mmi.rwth-aachen.de). Andre Kupetz is with the Dortmunder Initiative zur rechnerintegrierten Fertigung, Department Robot Technology, Dortmund, 44227 Germany (phone: +49-231-9700-778; fax: +49-231-9700-771; e-mail: kupetz@rt.rif- ev.de). Roland Wischnewski is with the Dortmunder Initiative zur rechnerintegrierten Fertigung, Department Robot Technology, Dortmund, 44227 Germany (e-mail: wischnewski@rt.rif-ev.de). interfaces. In this paper, we will present a novel framework for a flexible and intuitive user interface that is not restricted to a specific mobile robot. A. Motivation A large variety of events like earthquakes, gas-explosions, or acts of terrorism may lead to enormous damage to buildings and result in inaccessible and no-go areas for the rescue workers. The chance of survival in existing cavities is pretty high, but finding casualties is an extremely dangerous and intricate task because of the structural damage to the buildings and the lack of knowledge where to search. Therefore, the active exploration of the danger zone is a critical and necessary task for a precise localization and rescue operation. In doing so, the rescue worker exposes himself to life-threatening danger, because further collapses of the building are most likely. A robust mobile robot can support the rescue worker by exploring the damaged building, searching for victims, and giving information about the statics of the building. The operator should be able to easily navigate the robot and interpret the sensor data given by the robot. This offers specialists the possibility to evaluate the building’s structure. So, the rescue operation can be planned more efficiently and with less danger for the rescue worker. B. Scenarios A very common scenario is the occurrence of local damages. Typical examples of such scenarios are gas explosions in residential areas or acts of terrorism like bombing. Even large-area catastrophes like earthquakes can often be broken down into several spots of small operation areas. As a result, the rescue worker will concentrate on one special building. For a fast and efficient exploration of this area, the use of mobile robots is well suited. Figures 1 and 2 show possible scenarios for a rescue robot. Figure 1 illustrates a scenario that is commonly known as “horizontal layer”, in which the walls of a floor are collapsed but the ceiling is still intact. Furniture and other objects carry the ceiling so that a survival space exists. Figure 2 illustrates a scenario in which parts of floors are completely intact but the whole building is very close to collapse. Other examples are buildings that are leaning extremely to one side or buildings with severely damaged load bearing walls. All these scenarios have in common that the rescue worker has a relatively small area of exploration. An AR/VR based Approach towards the Intuitive Control of Mobile Rescue Robots Jürgen Roßmann, André Kupetz, and Roland Wischnewski T World Academy of Science, Engineering and Technology International Journal of Computer and Information Engineering Vol:4, No:11, 2010 1675 International Scholarly and Scientific Research & Innovation 4(11) 2010 scholar.waset.org/1307-6892/11540 International Science Index, Computer and Information Engineering Vol:4, No:11, 2010 waset.org/Publication/11540