1 Inspection of Complex Objects Using Multiple X-ray Views Domingo Mery, Member, IEEE Abstract—This paper presents a new methodology for identi- fying parts of interest inside of a complex object using multiple X-ray views. The proposed method consists of five steps: A) image acquisition, that acquires an image sequence where the parts of the object are captured from different viewpoints; B) geometric model estimation, that establishes a multiple view geometric model used to find the correct correspondence among different views; C) single view detection, that segment potential regions of interest in each view; D) multiple view detection, that matches and tracks potential regions based on similarity and geometrical multiple view constraints; and E) analysis, that analyzes the tracked regions using multiple view information, filtering out false alarms without eliminating existing parts of interest. In order to evaluate the effectiveness of the proposed method, the algorithm was tested on 32 cases (five applications using different segmentation approaches) yielding promising results: precision and recall were 95.7% and 93.9%, respectively. Additionally, the multiple view information obtained from the tracked parts was effectively used for recognition purposes. In our recognition experiments, we obtained an accuracy of 96.5%. Validation experiments show that our approach achieves better performance than other representative methods in the literature. Index Terms—X-ray testing, computer vision, tracking, auto- mated visual inspection, baggage screening. I. I NTRODUCTION X-ray imaging is used for both medical imaging and non-destructive testing (NDT) of materials and objects. The purpose of the latter application, called X-ray testing, is to analyze internal parts that are undetectable to the naked eye. X-ray radiation is passed through a test object and a detector senses variations in the intensity of the radiation exiting the object. The individual parts within an object can be recognized because they modify the expected radiation received by the sensor according to the differential absorption law [1]. There are numerous areas in which X-ray testing can be applied. In many of them, however, automated X-ray testing remains an open question and still suffers from: i) loss of generality, which means that approaches developed for one application may not transferred to another; ii) deficient detection accuracy, which means that there is a fundamental tradeoff between false alarms and miss detections; iii) limited robustness given that requirements for the use of a method are often met for simple structures only; and iv) low adaptiveness due to the fact that it may be very difficult to accommodate an automated system to design modifications or different specimens. This paper proposes a multiple view methodology for auto- mated X-ray testing that can contribute to reducing the four D. Mery is with the Department of Computer Science, Pontificia Universi- dad Catolica de Chile, dmery@ing.puc.cl, http://dmery.ing.puc.cl problems mentioned above. This methodology is useful for examining complex objects in a more general, accurate, robust and adaptive way given that this method analyzes an X-ray image sequence of a target object from several viewpoints automatically and adaptively. We observe that multiple view analysis has not yet been exploited in areas in which vision systems have typically been focused on single view analysis. This is the case of baggage screening, where certain items are very difficult to inspect from a single viewpoint because they could be placed in densely packed bags, occluded by other objects or rotated. For example, in Fig. 1d, it is clear that the part 1 (a pencil sharpener) could not be identified using a single (intricate) projection, however, it could be possible to recognize it if multiple projections of the part are available, as shown in Fig. 1e. Thus, multiple view analysis is used by our approach because it can be a powerful tool for examining complex objects in cases in which uncertainty can lead to misinterpretation. Its advantages are not limited to 3D interpretation, as two or more views of the same object taken from different points can be used to confirm and improve the diagnostic obtained by analyzing a single image. The main goals of our proposed multiple view methodology for detecting parts of interest in complex objects are: A) To acquire an image sequence where the parts of the object are captured from different viewpoints (Fig. 1a). B) To establish a multiple view geometric model used to find the correct correspondence among different views (lines in Fig. 1b). C) To segment potential regions (parts) of interest in each view using an application-dependent method that an- alyzes 2D features in each single view, ensuring the detection of the parts of interest (not necessarily in all views) and allowing for false alarms (points in Fig. 1c). D) To match and track potential regions based on similarity and geometrical multiple view constraints, eliminating those that cannot be tracked (lines in Fig. 1c). E) To analyze the tracked regions using multiple view in- formation, filtering out false alarms without eliminating existing parts of interest (see Fig. 1e where our approach is able to detect different parts recognizing for example a clip in 2 ). The main contribution of our work is a generic multiple X-ray view methodology that can be used to inspect complex objects in which the detection cannot be performed using a single view. The approach is robust for poor monocular seg- mentation and some degree of occlusion. In order to illustrate the effectiveness of the proposed method, the algorithm was