Condition Monitoring for Image-Based Visual Servoing Using Kalman Filter Mien Van, Denglu Wu, Shuzi Sam Ge, and Hongliang Ren (&) Advanced Robotic Centre, Faculty of Engineering, National University of Singapore, Singapore, Singapore ren@nus.edu.sg Abstract. In image-based visual servoing (IBVS), the control law is based on the error between the current and desired features on the image plane. The visual servoing system is working well only when all the designed features are cor- rectly extracted. To monitor the quality of feature extraction, in this paper, a condition monitoring scheme is developed. First, the failure scenarios of the visual servoing system caused by incorrect feature extraction are reviewed. Second, we propose a residual generator, which can be used to detect if a failure occurs, based on the Kalman filter. Finally, simulation results are given to verify the effectiveness of the proposed method. 1 Introduction Visual servoing, has been applied extensively in robotics to enhance sensing capability. The goal of this task is to calculate the control input that was applied to the robot system so that the predefined image features can converge to the desired static reference features. Generally, the visual servoing can be classified into three categories: (1) po- sition based visual servoing (PBVS) [1], where the control input is designed based on the feedback of 3D data such as the robotic system pose; (2) image-based visual servoing (IBVS) [2], where the control input is designed based on the feedback of 2D data defined in the image plane, and (3) hybrid visual servoing [3], where both 2D and 3D data is combined as the feedback. Among them, IBVS is widely applied due to it’s easy in implementation and robustness with calibration error and measurement noise [4–6]. In the IBVS, the control law is based on the different between the current and desired features on the image plane, which can be static for positioning problems or dynamic tracking problems. Because the control law is determined based on the image plane data, the system is working well only when all the designed features are correctly extracted. Toward this research direction, reliable feature extraction methods have been developed in computer vision, such as SIFT, SURF, BRIEF, KANSAC, etc. [7]. In addition, reliable feature tracking methods has also been developed to enhance the robustness of feature tracking based on kalman filter or Particle filter [15], etc. How- ever, the extraction of the designed image features is not always obtained correctly; some features will be appeared in or disappeared from the image during visual servoing [8]. Generally, there are two reasons making the appearance/disappearance of image features during visual servoing: (1) when the camera is moving, some parts of the © Springer International Publishing Switzerland 2015 G. Bebis et al. (Eds.): ISVC 2015, Part II, LNCS 9475, pp. 842–850, 2015. DOI: 10.1007/978-3-319-27863-6_79