Virtual Reality Tools for Internet Robotics Igor R. Belousov a Ryad Chellali Gordon J. Clapworthy Keldysh Institute of Applied Mathematics, Russian Academy of Sciences 4, Miusskaya Square, Moscow 125047, Russia belousov@spp.keldysh.ru l'Institut de Recherche en Communications et Cybernetique de Nantes (IRCCyN) 1, rue de la Noe, BP 92 101, 44321 Nantes France Ryad.Chellali@ircyn.ec-nantes.fr De Montfort University Dept. of Computer & Information Sciences Hammerwood Gate, Kents Hill, Milton Keynes MK7 6HP, United Kingdom gc@dmu.ac.uk a Author is supported by the INTAS grant YSF 99-4017 Abstract A virtual control environment for robot teleoperation via the Internet is presented. It comprises a Java3D-based real-time virtual representation of the robot and worksite, and uses a graphic panel, an environment for remote robot programming, and a dataglove with a 6D position tracker as the control interfaces. The use of Virtual Reality (VR) techniques for Internet teleoperation allow: (1) time delays inherent in IP networks to be suppressed, and (2) the operator's work to be simplified and accelerated, compared to methods that use delayed TV images. The system realisation, with its use of open technologies Java, Java3D and 3-tier client/server architecture, provides portability among different computer platforms and types of robots. The efficiency of the VR-based methods developed has been verified for slow communication rates (0.1-0.5 KB/sec), where TV- based control methods are inapplicable. VR systems have been developed for the WWW-based control of the PUMA and CRS industrial robot manipulators. The particulars of these systems, the experiments undertaken, current issues, and directions of future work are presented. 1. Introduction Robot control via the Internet is a growing and highly- promising branch of scientific research, industry and entertainment. Possible applications of Internet robotics are remote education, remote manufacturing [1], virtual visits to places of attraction (museums, parks, etc.) [2], and remote control of personal robots in the office or the home. Internet media have even been used for controlling objects in space (e.g. operations for the Mars polar lander mission, [3]). However, progress in this area has being hindered by current Internet limitations on communication bandwidth. Several systems have been under permanent on-line control over the last few years (the list of active systems providing free access through Web browsers is presented on the NASA Telerobotics Web-page [4]). While they provide the important possibility for all the Internet users to control the robots, these systems also have some disadvantages; in particular, none of them allows effective control because of the extremely slow replies by the system to the operator’s actions. Control within all of these systems is based purely only TV images, and these images are severely delayed (from a few seconds to several minutes). To avoid this limitation, methods of virtual 3D display of the robot and worksite should be applied. Some of the current Internet-based systems provide virtual displays of the current state of the robot. However, these are either pure 2D displays (for mobile robot control using 2D active maps, [2, 5]), or 3D displays that allow only simulation of the robot motion (e.g. for mission preparation), rather than real-time control [3]. Our goal was to use a dynamic, 3D virtual environment for real-time, on-line robot control. A 3D reconstruction of the robot's working environment was produced, and the robot, and the objects with which the robot interacts, were placed into it. The operator sends the control commands to the robot, and small data parcels containing the current coordinates of the robot and the objects are transmitted to the visualisation module at the operator’s site. These are then rendered in real time, allowing time delays to be suppressed (Fig. 1). This scheme considerably reduces system traffic as compared to TV-image transmission (see subsection 2.3 for estimation of the size of the data parcels). It allows the robots to be successfully controlled, even when communication rates are slow (0.1-0.5 KB/sec). Figure 1. System operation. This method has been tested for Internet-based control of the PUMA robot manipulator [6-8], and the mobile robot “Diligent” (Nomadic XR4400) [9] and has been shown to provide significantly-improved efficiency in the operator’s work. In this paper we present the latest results of the system-architecture development, new experiments on the control of the PUMA via Internet, a new system for Internet-based control of the CRS A465 robot manipulator, and experiments with this robot. The novelty of our system in the Internet robotics domain is also defined by the use of a tool for remote Proceedings of the 2001 IEEE International Conference on Robotics & Automation Seoul, Korea • May 21-26, 2001 0-7803-6475-9/01/$10.00© 2001 IEEE 1878