Large-scale Virtualized Reality I. Kitahara H. Saito S. Akimichi Center for TARA Dept. of Info. & CS Vision Media Studio University of Tsukuba Keio University Takenaka Corp. Tsukuba, 305-8577, Japan Yokohama, 223-8522, Japan Tokyo, 104-8182, Japan T. Onno Y. Ohta T. Kanade Inst. for Hyper Network Inst. of EMS Robotics Inst. Society University of Tsukuba Carnegie Mellon Univ. Oita, 870-0037, Japan Tsukuba, 305-8573, Japan Pittsburgh, PA. 15213, USA Abstract In this paper, we present our project for realizing Virtual- ized Reality in a large-scale space such as a soccer stadium, concert hall, etc. . We have developed an extended system of CMU 3D room [3] in an efficient way so that 4D digiti- zation of a large-scale space can practically realized. We place 16 progressive scan cameras around a large space (e.g. 70m 50m with 25m height). The videos from those cameras are digitized into PCs thatare connected via Ether- net. A camera calibration technique for a large-scale space that is accurate enough to apply CV-based algorithms is de- veloped. We also present a method to omit the calibration process with an idea of “Projective Grid Space”. The re- cent results and future works are presented for demonstrat- ing the significance of the project as a CV-based practical application. 1. Introduction One method for enhancing the visual effect is virtual move- ment of viewpoint so that the audience can virtually feel existence in the object scene. The recent applications of such effect are used in the Hollywood movie “Matrix”, in the SuperBowl XXXV broadcast by CBS as “EyeVision”, and so on. The effect is realized by taking a large number of synchronized multiple photos or videos and by switching them in order. Therefore, the observer cannot move every- where in the 3D space, but only on the locus of multiple real cameras. On the other hand, the development of 3D video display systems has become feasible with the progress of computer and video technologies, and several systems have been de- veloped [2][3][4][5]. Virtualized Reality [2] is one of the frontier project that realizes such virtual viewpoint move- ment for a dynamic scene by the use of CV technology. Al- though the “Matrix” and “EyeVision” are just a switching effect of multiple real camera images, the CV-based tech- nologies can generate completely novel 3D viewpoint im- ages for the virtual viewpoint movement effect. The size of the target 3D space handled in those systems, however, was relatively small because of the difficulty in recording the multiple videos and in calibrating the multiple cameras in a large-scale space. In this paper, we introduce our on- going project for realizing a Virtualized Reality system in a very large-scale space. 2. Large-scale Virtualized Reality Removing the size limitation of the Virtualized Reality technique (i.e., the size of the target 3D space) is to real- ize Virtualized Reality systems such as “a virtualized soccer game telecast in live” or “a virtualized recital in a concert hall on DVD”. Since, the observers can watch the dynamic events from completely novel 3D viewpoints, they can feel as if they are not only flying around but also going inside of the 3D space. Fig.2 illustrates an example of a view of an observer who virtually stands on the soccer field, which is generated based on our recently developed algorithm [1]. Storage media or broadcasting of the virtualized 3D data to other places provides an immersive presentation to the ob- servers beyond the time and distance. For such a practical virtualized reality system, we have to consider following is- sues. Digitized Recording of Multiple Videos In order to reconstruct an accurate 3D structure of the tar- gets, we need to capture multiple videos which satisfy the following conditions. Every frame of all videos is synchro- 1