IEEE JOURNAL zyxwvutsrqponmlk OF ROBOTICS AND AUTOMATION, VOL. RA-3, NO. zyxwvuts 1, FEBRUARY 1987 67 Communications Telerobotics: Display, Control, and Communication Problems LAWRENCE STARK, FELLOW, IEEE, WON-SO0 KIM, FRANK TENDICK, BLAKE HANNAFORD, MEMBER, IEEE, STEPHEN ELLIS, MEMBER, IEEE, MARK DENOME, MARY DUFFY, TIM HAYES, TED JORDAN, MARK LAWTON, TIM MILLS, ROBERT PETERSON, KATHLEEN SANDERS, MITCH TYLER, AND STEVEN VAN DYKE zyxwvutsr Abstract-An experimentaltelerobotics (TR) simulation is described suitable for studying human operator (HO) performance. Simple manipulator pick-and-place and tracking tasks allowed quantitative comparison of a number of calligraphic display viewing conditions. An enhancedperspectivedisplaywas effective withareferenceline from target to base, with or without a complex three-dimensional grid framing the view. This was true especially if geometrical display parameters such as azimuth (AZ) and elevation (EL) were arranged to be near optimal. Quantitative comparisons were made possible utilizing control performance measures such as root mean square error (rmse). There was a distinct preference for controlling the manipulator in end-effector Cartesian space for our primitive pick-and-place task, rather than controlling joint angles and then, via direct kinematics, the end-effector position. An introduced communication delay was found to produce decrease in performance. In considerablepart,thisdifficultycould be compensated for by preview control information. That neurological control of normal human movement contains a sampled data period of 0.2 s may relate to this robustness of HO control to delay. INTRODUCTION Manipulator control, enhancement of display, and effects of communication delays are three essential areas of research in telerobotics. Experiments in these areas were performed by students in the Biological Control Systems class (Mechanical Engineering 210) in the Mechanical Engineering Department at the University of California, Berkeley. Control of a manipulator in the end-effector space was compared with control of individual joint angles in pick- and-place tasks using a simulated manipulator. The value of enhancement of perspective display with a base grid and a reference line from target to base was measured in three-dimensional tracking experiments. The effect of communication delay was studied in one- dimensional pursuit tracking tasks, and preview was added to the display to measure the improvement in performance it allowed. All tasks were designed for simplicity and economy, yet were realistic enough to yield results that could be considered in design of the human-manipulator interface. Simultaneously, the members of the class were provided with an opportunityto study significant issues of telerobotics first-hand. Display An important display system goal in telerobotics is to provide an efficient man-machine interface with appropriate graphic and alpha- numeric presentation. Recent applications can be found in both Manuscript received September 6, 1985; revised April 18, 1986. This work was supported by the NASA-Ames Research Center under Cooperative Agreement NCC 2-86 and the Jet Propulsion Laboratory, California Institute of Technology, under Contract 956873. The authors are with the Departments of Physiological Optics, Engineering Science, and Neurology, University of California, Berkeley, CA 94720, USA. IEEE Log Number 8611970. telerobotics and cockpit displays. Pepper, Smith, and Cole [6] compared mono and stereo TV display systems for remote control of an undersea manipulator. Their experimental results showed that stereo TV displays are superior to mono TV displays under most conditions tested. The results also showed that as scene complexity and object ambiguity increased, the advantage of the stereo display became more pronounced. Future aircraft cockpits may be equipped with cockpit displays of traffic information. Since airplane traffic information is basically three-dimensional, careful studies are necessary on how to present such traffic information on two-dimensional display screens. Early cockpit displays were designed with plan-view projections, omitting. graphical representations of vertical information. Instead of adding numerical values or using different colors to indicate vertical information, Ellis and McGreevy [4] developed cockpit display systems based on perspective projections to provide a more natural format of representing three-dimensional information. Decoupled Manipulator When degrees-of-freedom of a manipulator are weakly coupled and motion is not too rapid, decentralized control of individual joints is possible [15]. The cylindrical manipulator configuration used in our simulated manipulator task had negligible coupling .of link dynamics, permitting simple direct and inverse kinematic calcula- tions. This kept the simulation real-time calculations simple. It also allowed human subjects to perform joint-angle control of the manipulator, with little practice, for comparison with end-effector control, where the computer calculated the required joint angles. Communication Delay Communication delay is an important constraint that may limit on- line control of a remote robot by human operator (HO). Data on the effect of delay on pursuit tracking is sparse, even though an established base exists in the numerous studies that have been made of human performance in one-dimensional pursuit tracking tasks [3], 151, [9], [I 11. It has been shown [7], [8], [13] that the HO can use preview information to improve tracking performance. Our group has performed pursuit tracking experiments with imposed delay and with compensating preview to measure their effects. METHODS Simulated Manipulator A real-time simulation program for a four degree-of-freedom cylindrical-type robot arm was developed for use in telerobotic control research. The program allows an operator, using joysticks, to control the motion of a simulated robot arm displayed in perspective on a high-speed digital calligraphic CRT display (Fig. 1). The simulation emulates a real system, wherein an operator may be remotely controlling a manipulator while viewing real or simulated images of the manipulator’s motion. Such systems would be typical of those which will be used in space in the construction and operation of space stations and in servicing of satellites in synchronous earth orbits. The simulation program is written in C and is implemented on an LSI-11/23 microcomputer with floating point hardware. The calli- graphic displayis interfaced to the computer through a 16-bit parallel inputloutput port. Joystick inputs are connected through 12-bit analog-to-digital converters. Two control modes, joint-angle control and end-effector Cartesian control mode, are implemented for the HO to control the robot motion. In joint-angle mode, four control axes of the two joysticks 0882-4967/87/0200-0067$01.0001987 IEEE