Position-based visual servo control of autonomous robotic manipulators Gangqi Dong, Z.H. Zhu n Department of Earth and Space Science and Engineering, York University, 4700 Keele Street, Toronto, Ontario, Canada M3J 1P3 article info Article history: Received 22 December 2014 Received in revised form 27 April 2015 Accepted 22 May 2015 Available online 7 June 2015 Keywords: Autonomous capture Non-cooperative target Extend Kalman filter Robotic manipulator Position-based visual servo On-orbit servicing abstract This paper concerns the position-based visual servo control of autonomous robotic manipulators in space. It focuses on the development of a real-time vision-based pose and motion estimation algorithm of a non-cooperative target by photogrammetry and extended Kalman filter for robotic manipulators to perform autonomous capture. Optical flow algorithm is adopted to track the target features in order to improve the image processing efficiency. Then, a close-loop position-based visual servo control strategy is devised to determine the desired pose of the end-effector at the rendezvous point based on the estimated pose and motion of the target. The corresponding desired joint angles of the robotic manipulator in the joint space are derived by the inverse kinematics of the robotic manipulator. The developed algorithm and position-based visual servo control strategy are validated experimentally by a custom built robotic manipulator with an eye- in-hand configuration. The experimental results demonstrate the proposed estimation algorithm and control scheme are feasible and effective. & 2015 IAA. Published by Elsevier Ltd. All rights reserved. 1. Introduction Robotic manipulators have been widely used in space for docking, assembling, repairing and other on-orbit servicing operations [1–5]. For instance, Mobile Servicing System (MSS) or Canadarm2 [6], Japanese Experiment Module Remote Manipulator System (JEMRMS) [7] and European Robotic Arm (ERA) [8] are typical examples of robotic manipulators performing assembly, maintenance, and payloads exchanging tasks on International Space Station. These operations were conducted either autono- mously or by human astronauts. Robotic manipulators mounted on Mars exploration rovers, such as, Viking 1 and 2 [9], Spirits and Opportunity [10], Phoenix [11] and Curiosity [12], were designed to collect soil samples and/or place instruments on a target. These tasks were performed by preprogrammed commands and controlled from the Earth directly or relayed by the Mars Orbiter. Cameras were used in these missions to monitor the movements of the manipulators and take photographs of the surround- ings. Robotic manipulators of orbital docking systems, such as the Shuttle Remote Manipulator System [13] and Orbital Express [14], performed tasks of grappling, dock- ing, refueling, repairing and/or servicing another space- craft. Pure experimental systems, such as, ROTEX (Robot Technology Experiment) and ETS-VII (Engineering Test Satellite) [15] demonstrated the operations of assembling, grasping, docking and exchanging orbit replaceable units by robotic manipulators. Most of these missions employed human-in-the-loop control. Manual control from the Earth may result in long time delay, while sending astronauts into space to perform the tasks suffers higher cost and the possibility of life loss. To address these challenges, auton- omous control is required and becomes a research high- light in the field of robotic technology [16,17]. Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/actaastro Acta Astronautica http://dx.doi.org/10.1016/j.actaastro.2015.05.036 0094-5765/& 2015 IAA. Published by Elsevier Ltd. All rights reserved. n Corresponding author. Tel.: þ1 416 7362100x77729. E-mail address: gzhu@yorku.ca (Z.H. Zhu). Acta Astronautica 115 (2015) 291–302