INVITED PAPER Calibrating Distributed Camera Networks Individual cameras estimate the position, orientation and focal length of neighboring cameras, and a distributed system-wide algorithm refines these estimates to produce accurate calibration. By Dhanya Devarajan , Zhaolin Cheng, and Richard J. Radke ABSTRACT | Recent developments in wireless sensor networks have made feasible distributed camera networks, in which cameras and processing nodes may be spread over a wide geographical area, with no centralized processor and limited ability to communicate a large amount of information over long distances. This paper overviews distributed algorithms for the calibration of such camera networks- that is, the automatic estimation of each camera’s position, orientation, and focal length. In particular, we discuss a decentralized method for obtaining the vision graph for a distributed camera network, in which each edge of the graph represents two cameras that image a sufficiently large part of the same environment. We next describe a distributed algorithm in which each camera performs a local, robust nonlinear optimization over the camera parameters and scene points of its vision graph neighbors in order to obtain an initial calibration estimate. We then show how a distributed inference algorithm based on belief propagation can refine the initial estimate to be both accurate and globally consistent. KEYWORDS | Belief propagation; bundle adjustment; camera calibration; distributed algorithms; metric reconstruction; sensor networks; structure from motion I. INTRODUCTION Modern urban life is characterized by the ubiquity of digital cameras. We are constantly imaged by our friends’ cell phones and digital cameras, the surveillance cameras in subway stations, busy streets, and shopping malls, and even mapping cameras on trucks or satellites. Many of these devices can now communicate wirelessly, so that the set of cameras can be viewed as a wide-area sensor network with thousands of nodes. Clearly, the proliferation of large numbers of interconnected cameras in the public sphere raises legitimate privacy concerns (e.g., see the paper by Widen in this Special Issue [92]). In contrast, here we are motivated by scenarios in which a camera network may be the best way to obtain time-critical information about an emergent situation where the safety or security of human lives is at stake, such as a natural disaster site, an urban combat zone, or a battlefield. Camera networks will be essential for twenty- first century military, environmental, and surveillance applications [1] but pose many challenges to traditional computer vision. Until recently, computer vision research on collections of tens or hundreds of cameras has generally taken place in a controlled environment with a fixed camera configura- tion. For example, many research labs have designed rooms in which the walls and ceiling are studded with cameras, for the purposes of three-dimensional (3-D) model acquisition and virtual reality (e.g., [2]–[4]). To undertake a computer vision task, images from all cameras are quickly communicated to a central processor. The same experimental assumptions clearly do not apply to real-world scenarios in which battery-powered cameras are spread over a wide geographical area. For example, camera nodes may be quickly deployed by soldiers moving rapidly through hostile terrain or first responders moving through a dangerous disaster zone. Accurate initial positions and orientations of such cameras will be unknown; even if some nodes are equipped with GPS receivers, these systems cannot be assumed to be highly accurate and reliable [5], nor do they operate indoors. The nodes are unsupervised after deployment and Manuscript received December 1, 2007; revised March 24, 2008. First published October 17, 2008; current version published October 31, 2008. This work was supported in part by the National Science Foundation under Award IIS-0237516. D. Devarajan was with the Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY 12180 USA. She is now with Immersive Media Company, Portland, OR 97214 USA. Z. Cheng was with the Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY 12180 USA. He is now with Captira Analytical, Albany, NY 12207 USA. R. J. Radke is with the Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY 12180 USA (e-mail: rjradke@ecse.rpi.edu). Digital Object Identifier: 10.1109/JPROC.2008.928759 Vol. 96, No. 10, October 2008 | Proceedings of the IEEE 1625 0018-9219/$25.00 Ó2008 IEEE