An Experimental Setup for Visual Servoing Applications on an Industrial Robotic Cell Vincenzo Lippiello, Bruno Siciliano, Luigi Villani Abstract— An experimental setup for visual servoing appli- cations on an industrial robotic cell is presented in this paper. The setup is composed of two industrial robot manipulators equipped with pneumatic grippers, a vision system and a belt conveyor. The original industrial robot controllers have been replaced by a single PC with software running under a real- time variant of the Linux operative system. A vision-oriented software environment named VESPRO has been developed on a PC running under Windows NT operating system, which allows programming image processing and visual tracking tasks, using one or more cameras. Advanced user interfaces permit fast, safe and reliable prototyping of control schemes based on visual measurements both for the single robots and for the dual-arm robotic cell. I. I NTRODUCTION The development of advanced sensor-based control algo- rithms for industrial robots requires open control architec- tures where software modules can be modified and extero- ceptive sensors like force/torque sensors and vision systems can be easily integrated. Various open control architectures for industrial robots have already been developed by robot and control manu- facturers as well as in research labs (see, e.g., [1], [2]). Most of them are based on a standard PC hardware and a standard operating system. In fact, a PC-based controller can more easily integrate many commercially available add-on peripherals and allows standard software development tools (e.g., Visual C++, Visual Basic, Delphi, etc.) to be used. An important issue of control software architectures deals with real-time operating systems. In recent years the hard real-time variants of the Linux operating system (RT- Linux [3] and RTAI-Linux [4]) are becoming widely adopted, especially in research labs [5], [6]. A common problem encountered in control architectures embedding visual measurements is that the time constraints of motion controllers are hardly met by the vision systems. This is especially true for position-based visual servoing [7], more than for image-based visual servoing [8]. In fact, the first approach requires computationally expensive operations to achieve the estimation of the pose (position and orienta- tion) of objects moving in the robot workspace. This problem is usually solved by adopting a so-called “indirect” visual servoing scheme [9], based on an inner/outer feedback loop where the inner position feedback loop runs at a frequency higher than the outer visual feedback loop, to guarantee The authors are with PRISMA Lab, Dipartimento di Informatica e Sistemistica Universit` a degli Studi di Napoli Federico II Via Claudio 21, 80125 Napoli, Italy {lippiell,siciliano,lvillani}@unina.it Fig. 1. The dual-arm industrial robotic cell at PRISMA Lab stability and disturbance rejection. This allows the time constraint of the visual feedback loop to be partially relaxed, so that non-hard real-time operating system can be adopted for the vision software. In this paper, an environment for visual servoing appli- cations on the industrial cell of PRISMA Lab, based on two robots Comau SMART-3 S, is presented. The control architecture is an open version of the industrial Comau C3G 9000, developed at the PRISMA Lab, which allows controlling both the robots using a standard PC working with RTAI-Linux operating system. The open controller, named RePLiCS [10], allows advanced control schemes to be designed and tested, including dual-arm cooperation, force control, as well as visual servoing. The visual system runs on a separate PC working with Windows NT operating system. A vision-oriented software environment named VESPRO has been developed to manage a multi-camera system and to perform visual pose estimation of objects moving in the cell. As an example of application, a position-based visual servoing task, involving both robots of the cell, is described. II. THE EXPERIMENTAL SETUP The setup in the PRISMA Lab consists of two industrial robots Comau SMART-3 S (see Fig. 1). Each robot ma- nipulator has a six-revolute-joint anthropomorphic geometry with nonnull shoulder and elbow offsets and non-spherical wrist. One manipulator is mounted on a sliding track which provides an additional degree of mobility. The joints are actuated by brushless motors via gear trains; shaft absolute resolvers provide motor position measurements. Proceedings of the 2005 IEEE/ASME International Conference on Advanced Intelligent Mechatronics Monterey, California, USA, 24-28 July, 2005 0-7803-9046-6/05/$20.00 ©2005 IEEE. WC2-01 1431