112 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 1, JANUARY 2015 Aircraft Recognition in High-Resolution Optical Satellite Remote Sensing Images Qichang Wu, Hao Sun, Xian Sun, Daobing Zhang, Kun Fu, and Hongqi Wang Abstract—Automatic aircraft recognition is a challenging task. Conventional methods always extract the overall shapes of aircraft at first and then represent the aircraft based on the extracted shape with different features for recognition. The major problem of these methods is that they have a high requirement on shape extraction, which is too idealistic for targets in satellite images. In this letter, we propose a new aircraft recognition approach that can recognize aircraft robustly without perfect extraction of silhouette or shape of aircraft as a precondition, and can deal with the situation of parts missing and shadow disturbance. Specifically, a direction estimation method is proposed first to align aircraft to a same direction. Then, a reconstruction-based similarity measure is proposed, which transforms the type recognition problem into a reconstruction problem. Finally, a jigsaw matching pursuit al- gorithm is proposed to solve the reconstruction problem. We use panchromatic Quickbird imagery for evaluation, and the experi- mental results illuminate that the proposed method is effective and accurate. Index Terms—Aircraft recognition, image processing, multi- scale segmentation, reconstruction. I. I NTRODUCTION T O RECOGNIZE the types of aircraft is very important. We can grasp the activity patterns of aircraft, detect the unusual trends of aircraft, and make judgment through type recognition. However, aircraft recognition with high-resolution spaceborne optical images is a challenging task. It is still difficult to distinguish targets of some types from the others. In conventional aircraft recognition methods, several meth- ods are based on using rotation-invariant features after binariza- tion. In [1], Hu moment invariant features are extracted from binary images to automatically identify six aircraft types. In [2], an independent component algorithm is combined with Zernike invariant moments for aircraft recognition. In [3], contour track- ing is used to eliminate much noise first and then uses moment invariants to recognize the types of the aircraft. These methods always use thresholding segmentation for the overall silhouette or shape of targets, and extract rotation-invariant features such as Hu moments, Zernike moments, wavelet moments, and Fourier descriptor for recognition. However, these methods have two drawbacks: 1) obtaining the moment invariants and Fourier descriptor requires perfect extraction of silhouette or Manuscript received January 9, 2013; revised May 22, 2013; accepted June 6, 2013. Date of publication June 24, 2014; date of current version August 14, 2014. (Corresponding author: H. Sun.) Q. Wu is with the College of Electronic Science and Engineering, National University of Defense Technology, Changsha 410073, China. H. Sun, X. Sun, D. Zhang, K. Fu, and H. Wang are with the Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China, and also with the Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Chinese Academy of Sciences, Beijing 100190, China (e-mail: sun.010@163.com). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LGRS.2014.2328358 shape of each aircraft as a precondition, which is too idealistic for targets with irregular appearance caused by distortion, low SNR, and camouflage painting in satellite images; and 2) these methods do not make full use of the shape characteristics of aircraft for target representation, which reflects the prior knowledge of the aircraft target, and will greatly enhance the robustness of recognition mission. In addition to the above kind of methods, there are also a few recognition methods that estimate the direction first after binarization and then recognize the types of aircraft [4]–[6]. These methods estimate the direction of aircraft before repre- senting targets that actually takes more aircraft shape charac- teristics, such as symmetry and fuselage characteristics, into account. These methods also require the binary image of each aircraft for direction estimation and the silhouette or contour with less fracture for target representation, which reduces the practicability of the above methods. As aircraft recognition is still a challenging problem, we want to further investigate how we can resolve issues in this field. Aircraft recognition is different from other natural object recognition: 1) the number of aircraft types is limited; and 2) each type of aircraft has fixed size and shape. Considering the above characteristics, we can build a template for each type and match the test aircraft to the different types of templates. By doing this, we can make more use of the shape characteristics of different types of aircraft. More importantly, we will focus on how to measure the similarity between targets and all types of templates, independent of the overall shape extraction of targets. In this letter, a novel type recognition approach for aircraft is proposed. In addition to making more use of the shape characteristics of different types of aircraft, the advantage of the approach lies in that it can recognize aircraft robustly without perfect extraction of silhouette or shape of targets as a precondition, and can deal with the situation of parts missing and shadow disturbance. The recognition approach consists of two steps: direction estimation and type recognition. In the ap- proach, a direction estimation method is proposed first to align aircraft to a same direction. Then, a reconstruction-based sim- ilarity measure is proposed, which transforms the type recog- nition problem into a reconstruction problem. Finally, a jigsaw matching pursuit algorithm is proposed to solve the problem. We evaluate the method using panchromatic 0.6-m-resolution Quickbird imagery, and the experimental results illuminate that the method proposed in this letter is effective and accurate. II. METHODOLOGY A. Direction Estimation Considering the shape characteristics of aircraft such as sym- metry and fuselage characteristics, we estimate the directions 1545-598X © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.