Aircraft type recognition in satellite images J.-W. Hsieh, J.-M. Chen, C.-H. Chuang and K.-C. Fan Abstract: This paper proposes a hierarchical classification algorithm to accurately recognise aircrafts in satellite images. Since each aircraft in satellite images is captured far from the ground, it has a very small size and often includes various textures, orientations, dazzle paints, and even noise. All of these will present many challenges in extracting useful features and result in unstableness and inaccuracy of aircraft type recognition. Therefore, before recognition, a novel symmetry-based algorithm is proposed to estimate an aircraft’s optimal orientation for rotation correction. In addition, several image preprocessing techniques such as noise removal, binarisation, and geometrical adjustments are also applied to removing the above variations. Then, distinguishable features are derived from each aircraft for aircraft recognition. However, different features have different discrimination abilities to recognise the types of aircrafts. Therefore, a novel booting algorithm is proposed to learn a set of proper weights from training samples for feature integration. Owing to this integration, significant improvements in terms of recognition accuracy and robustness can be achieved. Last, a hierarchical recognition scheme is proposed to recognise the types of aircrafts by using the area feature first for a rough categorisation on which detailed classifications are then achieved using several suggested features. Experiments were conducted on a wide variety of satellite images. Experimental results reveal the feasibility and validity of the proposed approach in recognising aircrafts in satellite images. 1 Introduction Satellite images can be captured without any constraints by time, weather, country boundary, and other environmental factors. Owing to this advantage, there have been many researchers who devoted themselves to utilising satellite images for developing different applications like water and climate observation, land cover classification, energy exploration, etc. Especially, surveillance through satellite images is another important application for military needs and environment protection. Therefore, in the literature [1–5], there have been many different detection schemes proposed for detecting various targets from satellite images such as bridges, airports, roads, streets, buildings, etc. For example, Nevatia and Babu [1] proposed a line edge detector to detect all line-like structures. Gruen and Li [2] used wavelet transforms to sharpen road boundaries. In addition, Geman and Jedynak [3] used an active testing strategy to detect all 1-D line structures as a base to find major roads. Moreover, Shi and Zhu [4] proposed a line matching method to extract road networks from high- resolution satellite images. In addition to line detection, Pesaresi and Benediktsson [5] used several morphological operations and the technique of multi-scale analysis to segment different buildings from satellite images. However, since the objects in satellite images are very small, all the above methods focus only on detecting objects and do not further recognise these objects. For recognising objects, in the past, there have been many methods [6–11] proposed for this task and requiring that the targets should be large enough for feature extraction. For example, Reeves et al. [6] and Wallace et al. [7] proposed procedures to identify a 3-D object from 2-D images using moments and Fourier descriptors. In addition, Tien and Chai [8] utilised the characteristics of non-uniform rational B-splines and cross- ratios to recognise aircraft in images. Greenberg and Guterman [9] used multi-layer neural networks to recognise different targets from aerial images according to the features of Zernike moments. Moreover, Moldovan and Wu [11] used a symbolic approach to recognise hierarchically aeroplanes if all features of an aeroplane were well extracted. However, when recognising the targets in satellite images, all these methods will fail to work since the analysed targets are very small and polluted by different dazzle paints, shadows, and other noise. In this paper, we propose a novel recognition system for recognising various aircraft in satellite images using a hierarchical boosting algorithm. Since each aircraft in satellite images has different orientations, sizes, textures, and even dazzle paints, before recognition, image pre- processing techniques are first employed to reducing all the above variations to a minimum. The preprocessing tasks include image quality enhancement, noise removal, auto- matic binarisation, and the adjustments of aircraft scaling and translation. For rotation correction, we propose a novel method to use the symmetrical property of an aircraft to estimate its optimal orientation. In the past, the common method to estimate an object’s orientation was through a moment-based analysis [12]. However, an aircraft may have longer wings, shadows, fragments, and other noise. All these factors will make the moment-based method fail to normalise an aircraft having a correct orientation. However, for an aircraft that has been fragmented, polluted, q IEE, 2005 IEE Proceedings online no. 20059020 doi: 10.1049/ip-vis:20059020 J.-W. Hsieh is with the Department of Electrical Engineering, Yuan Ze University, 135 Yuan-Tung Road, Chung-Li 320, Taiwan J.-M. Chen, C.-H. Chuang and K.-C. Fan are with the Department of Computer Engineering, National Central University, Jung-Da Rd., Chung- Li 320, Taiwan E-mail: shieh@saturn.yzu.edu.tw Paper first received 15th March and in revised form 25th November 2004 IEE Proc.-Vis. Image Signal Process., Vol. 152, No. 3, June 2005 307