Building footprint database improvement for 3D reconstruction: A split and merge approach and its evaluation Bruno Vallet ⇑,1 , Marc Pierrot-Deseilligny 1 , Didier Boldo 1 , Mathieu Brédif 1 MATIS Institut Géographique National 4, Av. Pasteur, 94165 Saint Mandé Cedex, France article info Article history: Received 19 August 2010 Received in revised form 22 June 2011 Accepted 26 June 2011 Available online 29 July 2011 Keywords: Photogrammetry Segmentation Reconstruction Urban scene Building abstract We present a general framework to improve a vectorial building footprint database consisting of a set of 2D polygons. The aim of this improvement is to make the database more proper to subsequent 3D build- ing reconstruction at a large scale. Each polygon is split into several simple polygons guided by a digital elevation model (DEM). We say that this segmentation is vectorial as we produce segmentations that intrinsically have simple polygonal shapes, instead of doing a raster segmentation of the DEM within the polygon then trying to simplify it in a vectorization step. The method is based on a Mumford and Shah like energy functional characterizing the quality of the segmentation. We simplify the problem by impos- ing that the segmentation edges have directions present in the input polygon over which the DEM is defined. We evaluate the validity of the proposed method on a very large dataset and discuss its pros and cons based on this evaluation. Ó 2011 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved. 1. Introduction 1.1. Problem statement The problem of urban reconstruction consists in finding a 3D model (in general a polyhedral surface) that is as coherent as pos- sible with the input data and has been widely studied over the past 15 years. Transferring the methods developed into industrial pro- duction at the scale of a large city raises robustness issues as a broad variety of building types may be encountered and a reason- able building model should be produced in a reasonable time in any case. In our experience, such robustness can only be achieved if a very reliable ground plan of the building is available. In this case, we can use the method of (Durupt and Taillandier, 2006). This method enumerates plausible models based on planes supported by the (horizontal) edges of the footprint, and selects the one that fits the data best. The robustness achieved by this method comes however at the cost of generality because it is limited to recon- structing continuous (C 0 ) surfaces. Unfortunately, the 2D databases available in France correspond to an administrative partition, and a single footprint often contains C 0 discontinuities (see Fig. 1), for instance, when:  Two (or more) adjacent buildings with different roof heights share the same footprint.  The real footprint of a building is only a portion of the footprint in the database (gardens, inner courts, etc.)  The building has some superstructures whose sizes and heights are not negligible with respect to the expected precision of the reconstruction. This problem becomes increasingly difficult as reconstructions gain in precision, and has already been tackled by Brédif et al. (2007) and Dornaika and Brédif (2008) in the context of photogrammetry. More difficult cases are often a combination of the three cited above, and require a manual intervention to enable a further reconstruction. In general, this intervention consists in subdivid- ing the footprint by cutting through all (or most of) the altimet- ric discontinuities. In a production framework, where large areas need to be extensively reconstructed, it appears that this build- ing footprint database enhancement step is the most time con- suming one. Hence, the problem that we tackle in this paper is that of automating this enhancement as a required pre-process- ing step to 3D reconstruction. More precisely, our problem is to segment a polygonal footprint into a set of non-overlapping polygonal sub-footprints that cover it entirely, such that the interface between the sub-footprints corresponds to altimetric discontinuities. This is a problem of segmentation of vector data (building footprints database) guided by raster data (photos, DEM, etc.) 0924-2716/$ - see front matter Ó 2011 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved. doi:10.1016/j.isprsjprs.2011.06.005 ⇑ Corresponding author. E-mail address: bruno.vallet@ign.fr (B. Vallet). URL: http://recherche.ign.fr/labos/matis (B. Vallet). 1 MATIS Lab/IGN. ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) 732–742 Contents lists available at ScienceDirect ISPRS Journal of Photogrammetry and Remote Sensing journal homepage: www.elsevier.com/locate/isprsjprs