752 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 53, NO. 2, FEBRUARY 2015 Robust Reconstruction of Building Facades for Large Areas Using Spaceborne TomoSAR Point Clouds Muhammad Shahzad, Student Member, IEEE, and Xiao Xiang Zhu, Member, IEEE Abstract—With data provided by modern meter-resolution syn- thetic aperture radar (SAR) sensors and advanced multipass interferometric techniques such as tomographic SAR inversion (TomoSAR), it is now possible to reconstruct the shape and mon- itor the undergoing motion of urban infrastructures on the scale of centimeters or even millimeters from space in very high level of details. The retrieval of rich information allows us to take a step further toward generation of 4-D (or even higher dimensional) dynamic city models, i.e., city models that can incorporate tempo- ral (motion) behavior along with the 3-D information. Motivated by these opportunities, the authors proposed an approach that first attempts to reconstruct facades from this class of data. The approach works well for small areas containing only a couple of buildings. However, towards automatic reconstruction for the whole city area, a more robust and fully automatic approach is needed. In this paper, we present a complete extended approach for automatic (parametric) reconstruction of building facades from 4-D TomoSAR point cloud data and put particular focus on robust reconstruction of large areas. The proposed approach is illustrated and validated by examples using TomoSAR point clouds generated from a stack of TerraSAR-X high-resolution spotlight images from ascending orbit covering an approximately 2-km 2 high-rise area in the city of Las Vegas. Index Terms—Clustering, facade reconstruction, point density, TerraSAR-X, tomographic SAR (TomoSAR) inversion, 4-D point cloud. I. I NTRODUCTION A UTOMATIC detection and reconstruction of buildings has been an active research area for at least two decades. Despite an extensive research effort, the topic is still of great interest due to ever increasing growth of urban population which gives rise to a wide range of potential applications in various fields. For instance, building footprints (i.e., 2-D building outline/shape) can be used for urban landscape devel- opment [1], urban planning [2], damage assessment [3], disaster management [4], navigation purposes [5], etc. Additionally, Manuscript received December 13, 2013; revised April 7, 2014; accepted May 7, 2014. This work was supported in part by the Helmholtz Associa- tion under the framework of the Young Investigators Group “SiPEO.” This work was also part of the project “6.08 4-D City” funded by the IGSSE of Technische Universität München and the German Research Foundation (DFG, Förderkennzeichen BA2033/3-1). M. Shahzad is with the Lehrstuhl für Methodik der Fernerkundung, Tech- nische Universität München, 80333 Munich, Germany (e-mail: muhammad. shahzad@bv.tum.de). X. Zhu is with the Lehrstuhl für Methodik der Fernerkundung, Technische Universität München, 80333 Munich, Germany and also with the Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), 82234 Wessling, Germany (e-mail: xiao.zhu@dlr.de). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TGRS.2014.2327391 2-D footprint data in combination with height information can generate 3-D building models. Such models are essential for virtual city modeling [6] and 3-D GIS applications (e.g., commercial software such as Google Earth, Apple Maps, etc.). Other possible usages may include analyzing solar potential over building roofs [7], placing and installing telecommunica- tion antenna towers [8], Web-based mapping [9], tourism [6], architecture [10], augmented reality applications [5], [11], and many more. Spaceborne synthetic aperture radar (SAR) sensors are able to provide day/night global coverage in virtually all weather conditions. Moreover, due to coherent imaging nature, by using acquisitions taken at different times, it is also uniquely capable of imaging the dynamics of the illuminated area in the scale of centimeters or even millimeters. These benefits have motivated many researchers, and therefore, several methods have been developed, which use very high resolution (VHR) spaceborne SAR imagery for detection and reconstruction of man-made structures in particular buildings. For instance, single-aspect SAR-image-based approaches for building reconstruction are proposed in [12]–[14]. Due to the fact that only single SAR images are used, these approaches predominantly perform well mostly only for isolated buildings but not for dense urban areas where the buildings are densely packed and smaller buildings are often occluded by the higher ones [15]. To resolve this, interferometric SAR (InSAR) data, SAR image pairs taken from slightly different viewing angles, are used, e.g., a modified machine vision approach is proposed in [16] to detect and ex- tract buildings. The algorithm is based on local approximation of best fitting planes in the segmented regions of interest. Simi- larly, Thiele et al. [17] also developed a model-based approach to detect and reconstruct building footprints using orthogonal InSAR images. Another automatic approach based on modeling building objects as cuboids using multiaspect polarimetric SAR images is presented in [18]. In data fusion context, the use of optical imagery has also been exploited along with SAR [19] and InSAR [15] datasets, respectively. Despite of the active ongoing research in the area, the problem of building recon- struction still remains challenging due to inherent characteris- tics with SAR images such as geometrical projection caused by the side-looking geometry [20]. Moreover, complex build- ing structures and high variability of objects appearing in the images make automatic building detection and reconstruction a challenging task. For example, problems posed by occlusion of smaller buildings/objects from the higher ones render diffi- culties in large area extension. Therefore, prior knowledge is often incorporated with certain regularization (geometric) con- straints for realistic and automatic reconstruction. For instance, 0196-2892 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.