JOURNAL OF EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS 1 Software Laboratory for Camera Networks Research Wiktor Starzyk † and Faisal Z. Qureshi ‡ , Member, IEEE Abstract—We present a distributed virtual vision simulator capable of simulating large-scale camera networks. Our virtual vision simulator is capable of simulating pedestrian traffic in different 3D environments. Simulated cameras deployed in these virtual environments generate synthetic video feeds that are fed into a vision processing pipeline supporting pedestrian detection and tracking. The visual analysis results are then used for subsequent processing, such as camera control, coordination, and handoff. Our virtual vision simulator is realized as a collection of modules that communicate with each other over the network. Consequently, we can deploy our simulator over a network of computers, allowing us to simulate much larger camera networks and much more complex scenes then is otherwise possible. Specifically, we show that our proposed virtual vision simulator can model a camera network, comprising more than one hundred active pan/tilt/zoom and passive wide field-of-view cameras, deployed in an upper floor of an office tower in downtown Toronto. Index Terms—virtual vision, camera networks, virtual reality I. I NTRODUCTION We envision that the next generation large scale smart cameras networks will be able to gather imagery over large areas, analyze this imagery, store it, and later retrieve it to support a multitude of applications ranging from surveillance and security to infrastructure management and entertainment with no or minimal human intervention. The sheer scale of these networks will render human intervention infeasible. Studying and designing such camera networks is a daunting task as problems in such disparate fields as image processing, embedded systems and networking must be addressed con- comitantly just to setup the basic experimental infrastructure. This observation led us to develop the virtual vision paradigm of camera networks research [1]. Virtual vision advocates using software tools capable of simulating camera networks under realistic scenarios to study and develop new camera networks algorithms. Virtual vision paradigm for computer vision research ad- vocates using visually and behaviourally realistic 3D envi- ronments, populated with pedestrians, automobiles, etc. to study camera networks. Virtual camera networks of suitable complexity can be simulated within these synthetic environ- ments, which we call virtual vision simulators. A virtual vision simulator offers several advantages over traditional physical W. Starzyk and F.Z. Qureshi are with the Faculty of Science, University of Ontario Institute of Technology, Oshawa, ON, L1H 7K4 Canada. † wiktor.starzyk@uoit.ca ‡ http://faculty.uoit.ca/qureshi Manuscript received X; revised X. Copyright c 2013 IEEE. Personal use of this material is permitted. How- ever, permission to use this material for any other purposes must be obtained from the IEEE by sending an email to pubs-permissions@ieee.org. Fig. 1: A view of our virtual world showing pedestrians walking on an upper floor of an office building. Toronto (Canada) skyline is visible through floor to ceiling panoramic windows. Our scripted pedestrians use motion-capture data to simulate realistic motion and cast dynamic shadows on the floor and the walls. camera setups during ideation, prototyping, and evaluation phases, including: • legal issues surrounding access to physical camera net- works installed in public spaces disappear when dealing with a simulated camera network; • developing a virtual vision simulator is a major under- taking; however, once such a simulator becomes avail- able, the cost of carrying out camera networks research within this simulator is minimal compared to performing research on a physical camera network—a virtual vision simulator runs on standard PCs and does not require any special hardware; • virtual vision offers quick prototyping—it is much easier and faster to reconfigure a virtual camera network than it is to reconfigure a physical camera network; • complex vision and control algorithms that need to be studied in “real time” can be easily studied in a virtual vision simulator by slowing down the virtual clock of the simulated environment; • virtual vision offers far faster design/evaluation iterations when compared to a physical camera network; • ground truth is readily available; and • camera control and coordination algorithms can easily be compared against each other since scenes are perfectly repeatable. Qureshi and Terzopoulos demonstrated a first virtual vision simulator of its kind in [1]. They deployed a network of 16 active pan/tilt/zoom (PTZ) cameras in a 3D reconstruction of the Penn train station. Here the Penn station was inhabited by up to 1000 self-animating pedestrians—tourists, commuters, etc.—that look and behave like real humans [2]. They used this simulator to study camera scheduling and coordination