514 IEEE TRANSACTIONS ON AUTOMATIONSCIENCE AND ENGINEERING, VOL. 6, NO. 3, JULY 2009 Automated Refinement of Automated Visual Inspection Algorithms Hugo C. Garcia and J. Rene Villalobos Abstract—One of the challenges faced by the users of automated visual inspection (AVI) systems is how to efficiently upgrade the legacy systems to inspect the new components introduced into the assembly lines. If the AVI systems are not flexible enough to accom- modate new components, they will be rendered obsolete even by small changes in the product being inspected. The overall objective of the research presented in this paper is to produce the method- ological basis that will result in the development of highly reconfig- urable AVI systems. In this paper, we focus on part of this overall development, the adaptation of preexisting inspection algorithms to inspect similar components introduced into the assembly line. While this paper bases its development and discussion on the in- spection of surface mounted devices (SMDs), the proposed method- ology is general enough to be applicable to a broad range of inspec- tion problems. Note to Practitioners—In this paper, we present a methodology that would allow the automation of the refinement of AVI algo- rithms. In particular, the proposed method identifies a set of com- ponents, or cluster of components, for which a particular set of in- spection features or algorithms, renders a certain level of inspec- tion reliability. This is particularly useful for adapting preexisting systems to inspect new components, especially when the character- istics of the new components are similar to those of components al- ready inspected by the inspection system. We applied this method- ology to a case of study of the inspection of SMDs. Index Terms—Automated visual inspection (AVI) system, quadratic classification function (QCF), surface mounted devices. I. INTRODUCTION O NE of the main problems with automated visual inspec- tion (AVI) systems is how to upgrade the legacy system to be capable of inspecting new components introduced into the assembly line. A major problem of the existing inspection sys- tems is the high reconfiguration cost, resulting from hardware and software development, and labor-maintenance costs [1] and [2]. Many of the current AVI systems are custom-designed for a specific task and often are very hard to adapt to new applications [3]. Moreover, the reconfiguration process is a difficult task be- cause it requires knowledge in many technical areas including Manuscript received October 04, 2007. First published May 15, 2009; cur- rent version published July 01, 2009. This paper was recommended for pub- lication by Associate Editor Y. F. Li and Editor M. Wang upon evaluation of the reviewers’ comments. This work was supported in part by the National Sci- ence Foundation under Grant DMI-0300361 for the realization of this research project. H. C. Garcia was with the Department of Industrial Engineering, Arizona State University, Tempe, AZ 85281 USA. His is now with L3, Electro-Optical Systems, Tempe, AZ 85281 USA (e-mail: Hugo.Garcia@L-3com.com). J. R. Villalobos is with the Department of Industrial Engineering, Arizona State University, Tempe, AZ 85281 USA (e-mail: Rene.Villalobos@asu.edu). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TASE.2009.2021354 illumination, optics, image processing, and real-time program- ming [4]–[6]. Finding the personnel with expertise in all these areas is a difficult and expensive task [6]. As a consequence if, the AVI systems do not have enough flexibility to accommodate new products, these systems will be rendered obsolete even by small changes in the product being inspected. This problem is particularly intense in the electronics industry where the intro- duction of new types of components or products is almost a daily occurrence. Thus, it is necessary to develop reconfigurable AVI systems that are easily adaptable to new products to extend their economic life and to reduce the per-unit inspection cost. Ideally, the adaptation or reconfigurability of the inspection algorithms to inspect new components should be automated, built into the inspection system, and transparent to the human user. The overall objective of the research presented in this paper is to produce a methodological basis that will result in the emer- gence of highly reconfigurable AVI systems. In this paper, we focus on part of this overall development, the adaptation of pre- existing inspection algorithms to inspect similar components in- troduced into the assembly line. In the first part of this paper, we present a methodology that will allow the automatic formation of clusters for which a particular inspection algorithm renders the desired inspection performance. This is important because this methodology will speed up the development of inspection algorithms by allowing the incremental adaptation of the inspec- tion algorithms to different cluster of components by changing the appropriate algorithms’ parameters. In the second part of this paper, we introduce a method to estimate the performance of the inspection algorithms being developed during the actual inspection process. This method has the potential of shortening the process of adapting the inspection algorithms by screening out those algorithms that may have performed well in the devel- opment phase but may not do so in the actual inspection. We believe that while this paper bases its development and discussion on the inspection of surface mounted devices (SMDs), the methodology is general enough to be applicable to a broad range of inspection problems. In a previous paper, [7], the authors laid out the foundations for the inspection methodology used in this research. In par- ticular, they introduced an iterative vector-based approach for the inspection of SMD components. In this approach, for each component being inspected, a fixed number of inspection fea- tures are extracted. Then, a real value is associated with each feature; these values form a vector that is used for the classifi- cation of the SMD under inspection as present or absent using the quadratic classification function (QCF). Using this previous work as a starting point, we present an iterative methodology to facilitate the automated adaptation of preexisting inspection sys- tems. The general steps of the methodology are given in Fig. 1. 1545-5955/$25.00 © 2009 IEEE