To appear in Proc. IEEE Computer Vision and Pattern Recognition, December 2001. Calibrated, Registered Images of an Extended Urban Area Seth Teller Matthew Antone Zachary Bodnar Michael Bosse Satyan Coorg Manish Jethwa Neel Master MIT Computer Graphics Group Abstract We describe a dataset of several thousand calibrated, geo-referenced, high dynamic range color images, ac- quired under uncontrolled, variable illumination in an outdoor region spanning hundreds of meters. All im- age, feature, calibration, and geo-referencing data are available at http://city.lcs.mit.edu/data. Calibrated imagery is of fundamental interest in a wide variety of applications. We have made this data available in the belief that researchers in computer graphics, computer vision, photogrammetry and digital cartography will find it useful in several ways: as a test set for their own algorithms; as a calibrated image set for applications such as image-based rendering, met- ric 3D reconstruction, and appearance recovery; and as controlled imagery for integration into existing GIS systems and applications. The web-based interface to the data provides inter- active viewing of: high-dynamic-range images and mo- saics; extracted edge and point features; intrinsic and extrinsic calibration, along with maps of the ground context in which the images were acquired; the spatial adjacency relationships among images; the epipolar ge- ometry relating adjacent images; compass and absolute scale overlays; and quantitative consistency measures for the calibration data. 1. Introduction This paper describes data produced by a system for calibrated, terrestrial image acquisition in urban areas. The system is end-to-end, in the sense that it acquires uncalibrated images as input, and produces accurately calibrated, geo-referenced images as output, with no human interaction other than operation of the sensor. Four key ideas distinguish our approach [27] from other methods. Every image is annotated with an ab- solute, GPS-based position estimate as it is acquired, enabling efficient discovery of adjacent (and likely re- lated) images in subsequent processing. The sensor acquires omni-directional images for more robust re- covery and accurate estimation of scene structure and camera motion. Probabilistic, projective uncertainty models are used throughout the system to represent noisy features and camera pose. Finally, all of the sys- tem’s algorithmic components have asymptotically lin- ear time and space requirements, enabling their appli- cation to very large datasets. Detailed descriptions of the system, sensor, and processing components can be found elsewhere [25, 26, 11, 12, 10, 6, 7, 27]. An extensive collection of calibrated, high dynamic range (HDR) image data produced by the system is now available for interactive viewing and download at http://city.lcs.mit.edu/data. We envision at least three ways in which the data may be useful to other researchers. First, the uncalibrated imagery (i.e., raw sensor data) can be used as a test dataset by other researchers developing intrinsic or extrinsic calibration methods. Second, the registered imagery (i.e., data produced by our registration algorithms) can be used in a variety of applications including image-based ren- dering, scene reconstruction and texture estimation. In either context, the scale and extent of the data should pose an interesting collection of challenges. Finally, geo-referenced (Earth-relative) extrinsic calibration en- ables the imagery to be readily incorporated into a vari- ety of existing GIS and digital cartography applications (e.g. OpenGIS [4], TerraServer [5], and the National Spatial Data Infrastructure [3]). The paper is organized as follows. Section 2 de- scribes the acquisition and processing stages used to produce the dataset. Section 3 describes several quan- titative consistency measures for the image metadata. Section 4 describes a web interface to the dataset. (In- formation about file formats and organization is de- ferred to an Appendix, and documented on-line.) Sec- tion 5 describes existing acquisition methods for geo- referenced imagery. Section 6 summarizes the contri- butions of the paper. Proceedings of IEEE Computer Vision and Pattern Recognition, December 2001