Control and Intelligent Systems, Vol. 33, No. 3, 2005 A REPETITIVE SEGMENTED TRAINING NEURAL NETWORK CONTROLLER WITH APPLICATIONS TO ROBOT VISUAL SERVOING P. Jiang ∗ and Y. Chen ∗∗ Abstract The authors design a neural network controller for a nonlinear system with uncertainties that are invariant or repetitive over repeatedly executed tasks. The training of the neural networks is carried out iteratively as the task repeats. The desired trajectory is segmented, and for each segment a local neural network is constructed to keep the training errors within a permitted region. Meanwhile, the training is segment-wise progressively from the starting segment to the ending one. The accurate tracking of the whole desired trajectory is thus accomplished in a step-by-step or segment-by- segment manner, which means that the training of the second segment starts after the first segment tracking has reached a desired accuracy level. To guarantee the uniform boundedness of the point- wise training, a projection-type learning update law and deadzone technique are proposed. As an application example, a robot visual servoing control problem with an uncalibrated camera is considered. The effectiveness of the proposed neural network controller with repetitive segmented training is demonstrated by simulations of a typical robot visual servoing task, robot movement learning from demonstration. Key Words Neural networks, iterative learning control, nonlinear control, visual servoing 1. Introduction Ordinarily, there are two different methods that can be used to cope with the control problem of an uncertain or unknown nonlinear system: linear adaptive control [1, 2] and robust control [3, 4]. Generally speaking, robust control uses a sufficient feedback to restrain the influence of uncertainties and linear adaptive control uses an adap- tive feedforward mechanism to compensate uncertainties. ∗ Department of Cybernetics and Virtual Systems, University of Bradford, Bradford, BD7 1DP, UK; e-mail: ∗∗ Department of Electrical and Computer Engineering, UMC 4160, College of Engineering, 4160 Old Main Hill, Utah State University, Logan, UT 84322-4160, USA; e-mail: (paper no. 201-1344) Compared with robust control, linear adaptive control re- lies more on prior model knowledge, for example, linearily parameterized model. In order to control a nonlinear sys- tem with an unknown model structure, neural networks (NN) have been introduced to learn and reconstruct the unknown nonlinearities (e.g., GRBF NN [5], multilayer NN [6], CMAC NN [7], and Fuzzy NN [8]). These approaches often worked by combining the linear adaptive control technique with the robust control technique. Whereas NNs are used to approximate uncertainties with only unknown linear weights, the linear adaptive technique is adopted to update the weights and the residual modelling errors are controlled by a robust control scheme. In the approaches just mentioned, the desired oper- ation was supposed to be continual and the asymptotic tracking was achieved through persistent training along time horizon. For a nonperiodic trajectory with a finite time interval, which is widely applied in practical systems such as cutting, painting, robot pick-and-place operation, and multi-times resetting, tracking becomes a choice to keep the training persistently. However, multi-times reset- ting violates the assumption of continual operation along the time horizon of the linear adaptive control. Iterative Learning Control (ILC) [9] is a more specialized technol- ogy for this kind of application and can be considered an efficient alternative. In this article, we present an NN iterative training scheme for tracking a nonperiodic trajectory with a finite time interval based on ILC. The NNs are used as approx- imators of unknown nonlinearities instead of the common structure assumptions in ILC, such as meeting a global Lipschitz condition [10–13], with known linearly param- eterized nonlinearities [14, 15]. Unlike the adaptive NN controls, we give every point (every servo period in appli- cation) along the desired trajectory an independent local NN, and the independent networks are trained according to the previous tracking history. This makes segmented training possible because the training of any segment of a trajectory does not interfere the other segments. In com- mon with neural network–based controls, during the early 1