A SEMI-AUTOMATIC PROCEDURE FOR TEXTURING OF LASER SCANNING POINT CLOUDS WITH GOOGLE STREETVIEW IMAGES J. F. Lichtenauer a, ∗ , B. Sirmacek b a Laan der Vrijheid 92, 2661HM Bergschenhoek, The Netherlands - jeroenlichtenauer@gmail.com b Department of Geoscience and Remote Sensing, Delft University of Technology, Stevinweg 1, 2628CN Delft, The Netherlands - www.berilsirmacek.com Commission VI, WG VI/4 KEY WORDS: Point Clouds, Terrestrial Laser Scanning, Structure from Motion (SfM), Texturing, Google Streetview ABSTRACT: We introduce a method to texture 3D urban models with photographs that even works for Google Streetview images and can be done with currently available free software. This allows realistic texturing, even when it is not possible or cost-effective to (re)visit a scanned site to take textured scans or photographs. Mapping a photograph onto a 3D model requires knowledge of the intrinsic and extrinsic camera parameters. The common way to obtain intrinsic parameters of a camera is by taking several photographs of a calibration object with a priori known structure. The extra challenge of using images from a database such as Google Streetview, rather than your own photographs, is that it does not allow for any controlled calibration. To overcome this limitation, we propose to calibrate the panoramic viewer of Google Streetview using Structure from Motion (SfM) on any structure of which Google Streetview offers views from multiple angles. After this, the extrinsic parameters for any other view can be calculated from 3 or more tie points between the image from Google Streetview and a 3D model of the scene. These point correspondences can either be obtained automatically or selected by manual annotation. We demonstrate how this procedure provides realistic 3D urban models in an easy and effective way, by using it to texture a publicly available point cloud from a terrestrial laser scan made in Bremen, Germany, with a screenshot from Google Streetview, after estimating the focal length from views from Paris, France. 1. INTRODUCTION In recent years, a lot of progress has been made with the devel- opment of three dimensional (3D) sensors, 3D visualisation tech- nologies and data storage-, processing- and distributions possi- bilities. This seems good news for one of its most useful ap- plications: 3D mapping of our urban environments. However, the main limitation still left for large-scale realistic digitization of urban environments is the collection of data itself. Although flying- and Earth-orbiting cameras and 3D sensors allow cover- age of the entire globe at relatively low cost, a view from above is severely restricted by occlusions. Unfortunately, the most rel- evant urban areas for daily use are mostly at ground level, rather than on rooftops or above tree canopies. This means that terrestrial short-range measurements still need to be made to map urban environments realistically in 3D. While state-of-the-art mobile laser scanners can capture high quality 3D data together with coloured texture, their prices are high and the acquisition of texture data often requires significant extra scan- ning time over the capture of 3D data alone. It is mainly because of the cost of these sensors and the time it takes to use them to ac- quire measurements from all the required locations, that realistic 3D urban maps are still not widely available today. Therefore, in order to make full use of the available 3D data tech- nologies for realistic urban mapping, we also need to be able to include data from fast, low-cost scanning methods, as well as from previously recorded data, which do not always include colour texture information together with the 3D measurements. And even with 3D scans that include colour texture, unsatisfac- tory lighting conditions during a scan might still require new tex- ture to be added afterwards. ∗ Corresponding author To overcome this limitation, we propose a simple semi-automatic method to accurately texture 3D data with photographs, which consists of a novel combination of currently available free soft- ware implementations of pre-existing algorithms for Structure from Motion (SfM), camera pose estimation and texture map- ping. Only three or more tie points per photo are needed to ac- curately align it to the corresponding 3D data. These tie points can either be obtained from a simple manual procedure that can be done by anyone without special training, or from an automatic 2D to 3D feature matching algorithm. To solve the problem of calibrating a panoramic viewer without being able to use an a priori known calibration pattern, we propose to obtain intrinsic camera parameters from a Structure from Motion algorithm ap- plied to any suitable data that is already available from the viewer. To demonstrate that our methods works effectively, we applied it to texture a laser scan of building faades at the Bremer Market Square in Germany with a screenshot from Google Streetview, using a focal length estimation made from views of the ’Arc de Triomphe’ in Paris, France. The results indicate that our method allows to easily make realistic-looking digital urban 3D mod- els by combining pre-recorded data from completely unrelated sources. 2. PREVIOUS WORK A promising way to easily generate photo-realistic urban 3D mod- els is the use of Structure from Motion (SfM). Snavely et al. (2008) have shown that photos from the internet can be used with SfM to automatically reconstruct buildings and small parts of cities. This produces fully textured 3D models without even having to visit a site. Unfortunately, it requires having at least 3 good quality photos available on the internet of every build- ing that needs to be reconstructed. This limits the application of The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-3/W3, 2015 ISPRS Geospatial Week 2015, 28 Sep – 03 Oct 2015, La Grande Motte, France This contribution has been peer-reviewed. Editors: U. Stilla, F. Rottensteiner, and S. Hinz doi:10.5194/isprsarchives-XL-3-W3-109-2015 109