Interactive panorama video viewer with head tracking algorithms Yu-Jin Hong, Jae-In Hwang, Sun-Bum Youn, Sang Chul Ahn, Hyung-Gon Kim, Heedong Ko Imaging Media Research Center Korea Institute of Science and Technology (KIST) Seoul, Korea hyj@imrc.kist.re.kr Abstract—This paper presents new interaction method to control panorama video using head tracking technique. The panorama video viewer includes head tracking module which controls the rotation and forward/backward of the viewpoint. We applied this interaction technique to panorama video, which has omnidirectional gaze, to maximize realistic and immersive feelings. The proposed panorama video system was applied to the baseball broadcasting to show the feasibility of the suggested interaction technique. Keywords-component; HCI; Computer Vision; Head tracking; Input Devices; Panorama viewer I. INTRODUCTION With the new media contents and information technology, more intuitive and friendly methods for human computer interaction (HCI) are being developed which do not rely on traditional devices such as mouse, keyboards or joysticks. These interaction researches are focused on how to convey user’s intensions to the computer effectively. Therefore, many researchers performed studies of classifying user’s body movement due to its immediate response. In recent years there has been growing interest in a class of methods based on computational vision in a natural way [1]. Project Natal [2] uses human body gestures as inputs, without any other controller devices but with the depth camera. However body movements can be widely used for representing general intensions but not a good method to perform detailed actions. Therefore, face and hand tracking methods are introduced to include more complicated and exact intensions. J. Segen and S. Kumar [3] uses a controlled background to localize the hand efficiently in real-time. J. Yang and A. Waibel [4] presented a real-time face tracker using camera with a stochastic skin-color model that can be applied to lip-reading and gaze tracking. Francois [5] showed the possibility head motion as a new input stream which has the ease of learning. L.P. Morency and T. Darrell [6] developed vision-based head gesture interface module without conventional input devices and showed the performance and feasibility of the system through user studies. Panoramic image has omnidirectional view points, thus, many applications are developed its viewers for touring since it supports immersive feelings. Photosynth [7] can generate panoramic image using stitching technique and panorama viewer [8] displays and navigates with mouse or keyboard as traditional control device. In our study, human’s head movement plays dominant input to reacts directly user’s action. In addition, we employ this module as 360-degree panorama video viewer to maximize vividness and immersion. II. DESIGN A. System Overview Fig. 1 shows the overall structure for our system. First, we capture wide-angle photos with spherical camera [9] which is wired to PC. These photos are stitched and used as source of panorama viewer. In order to control panorama viewer in directly with no need to touch a controller such as the mouse or the keyboard, we calculated 3-D head position and orientation tracking module [10] with USB-web camera in real-time. Figure 1. System Structure 978-1-4244-7570-4/10/$26.00 ©2010 IEEE