IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 13, NO. 3, JUNE 2008 393 Population-Based Uncalibrated Visual Servoing Mirjana Bonkovi´ c, Aleˇ s Hace, and Karel Jezernik, Senior Member, IEEE Abstract—This paper introduces the implementation of a re- cently introduced method suitable for visual servoing. The method is based on the generalization of secant methods for nonlinear opti- mization. The difference with existing approaches related to visual servoing is that we do not impose a linear model to interpolate the goal function. Instead, we prefer to identify the linear model by building the secant model using population of the previous it- erates, which is as close as possible to the nonlinear function, in the least-squares sense. The new system has been shown to be less sensitive to noise and exhibits a faster convergence than do conven- tional quasi-Newton methods. The theoretical results are verified experimentally and also by simulations. Index Terms—Broyden–Jacobian estimation, fixed imaging, nonlinear optimization, population-based generalization, uncali- brated systems, visual servoing. I. INTRODUCTION T HE THEORY and practice of visual servoing has reached the level of maturity where excellent results can be achieved for various kinds of problems [3]–[8]. Visual servoing problem is, in fact, the problem of solving systems of nonlinear equations, which is a mature and well-founded mathematical area with rich foundations of papers [9]–[16], studying the vari- ous types of problems with appropriate approaches. It is impor- tant to emphasize that the achievements in that field, which are regularly and theoretically well founded, are not often utilized for practical solutions due to a gap that exists between various research fields. For technical solutions, engineering practice has usually relied on a conventional optimization technique based on the least-squares methods. If the applied approach does not meet the defined requirements, then alternative techniques such as the neuro-fuzzy-genetic approach or appropriate ad hoc constructed adaptive techniques usually result in optimized solutions that function well for a given problem. However, some solutions are often not theoretically based due to which the method cannot be established as a reference for similar problems. In this paper, we have shown that the advances in the standard optimization algorithms (e.g., the system of nonlinear equations solving), which are well founded and proved in theory, could be successfully used for typical technical problem solving. The word “typical” in this context fits well to uncalibrated visual Manuscript received May 20, 2007; revised November 4, 2007. Recom- mended by Technical Editor Y. Hori. This work was supported in part by the Slovenian Ad Futura Science Agency under a scholarship grant and in part by the Croatian Ministry of Science under Project 023-0232005-2003. M. Bonkovi´ c is with the Faculty of Electrical Engineering, Mechanical Engi- neering and Naval Architecture, University of Split, 21000 Split, Croatia (e-mail: mirjana.bonkovic@fesb.hr). A. Hace and K. Jezernik are with the Institute of Robotics, Univer- sity of Maribor, SI-2000 Maribor, Slovenia (e-mail: ales.hace@uni-mb.si; karel.jezernik@uni-mb.si). Color version of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TMECH.2008.924135 Fig. 1. Uncalibrated IBVS control scheme. servoing control problem [1], [2] since the plant is nonlinear, highly coupled, and influenced by the noise. The next section of the paper presents what our contribu- tion(s) are over the previous methods, together with a brief pre- sentation of the population algorithm. Section III is dedicated to simulations, while Section IV describes the experimental re- sults. Section V concludes the paper. II. UNCALIBRATED VISUAL SERVOING A. Image-Based Visual Servoing (IBVS) The main goal of visual servoing is to move the end-effector to a certain pose with respect to particular object or features in the image. Particularly, we are interested in a image-based visual servoing system (IBVS) (Fig. 1), the specification of which involves determining an appropriate error function e, such that when the task is achieved, e = 0 [1]. As the robot control input is usually defined in joint or in task space coordinates, it is necessary to relate the changes in visual appearance ( ˙ f ) with the changes in joint or task space ( ˙ q) through the Jacobian matrix J (q) ˙ f = J (q)˙ q. (1) Let the joint coordinates be control input to the robot system. Then, the Jacobian matrix J is a compound of robot and image Jacobians given by J = J I J R . (2) The depth estimation and the number of feature parameters re- lated to the number of degrees of freedom the robot has to control could be identified based on camera and robot calibra- tion parameters. The identification of the accurate values of the compound Jacobian matrix, which is, in general, a nonlinear function of q, is a hard problem. However, the problem could also be solved using a model-free approach that describes visual servoing algorithms that are independent of hardware (robot and camera systems) types of configuration. In this case, the prob- lem is considered as a problem of uncalibrated visual servoing in which the true Jacobian matrix is replaced with its model 1083-4435/$25.00 © 2008 IEEE