Stereotime: A Wireless 2D and 3D Switchable Video Communication System You Yang 1 , Qiong Liu 1 , Yue Gao 2 , Binbin Xiong 1 , Li Yu 1 , Huanbo Luan 2 ,Rongrong Ji 3 , Qi Tian 4 1. Department of Electronics & Information Engineering, Huazhong University of Science & Technology, Wuhan, China 2. School of Computing, National University of Singapore, Singapore 3. Department of Cognitive Science, School of Information Science & Engineering, Xiamen University, Xiamen, China 4. Department of Computer Science, University of Texas at San Antonio {yangyou,q.liu,hustlyu}@hust.edu.cn, {kevin.gaoy,viclol36,jirongrong}@gmail.com, qitian@cs.utsa.edu ABSTRACT Mobile 3D video communication, especially with 2D and 3D compatible, is a new paradigm for both video commu- nication and 3D video processing. Current techniques face challenges in mobile devices when bundled constraints such as computation resource and compatibility should be con- sidered. In this work, we present a wireless 2D and 3D switchable video communication to handle the previous chal- lenges, and name it as Stereotime. The methods of Zig-Zag fast object segmentation, depth cues detection and merging, and texture-adaptive view generation are used for 3D scene reconstruction. We show the functionalities and compatibil- ities on 3D mobile devices in WiFi network environment. Categories and Subject Descriptors H.4.3 [Information Systems Applications]: Communi- cations Applications; H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems Keywords Video communication, Visual content, View generation, Scene reconstruction 1. INTRODUCTION Video communication has changed people’s daily life for instance providing face-to-face long distance communication [1]. With the development of portable devices, video com- munication becomes ubiquitous, with popular and practical products include Facetime, LinPhone, etc. In recent years, three dimensional (3D) technologies [2, 3] have been widely applied. 3D displaying has been ported to mobile devices, and brings people with immersive 3D viewing experiences. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prof t or commercial advantage, and that copies bear this notice and the full ci- tation on the f rst page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). Copyright is held by the author/owner(s). http://–enter the whole DOI string from rightsreview form conf rmation. Figure 1: An overview of our proposed 3D Facetime. With such attractive displaying capacity, 2D and 3D switch- able functionality becomes a new paradigm in the fields of video communication and 3D video processing. A natural solution is porting the system directly from wired networks to wireless networks and mobile devices [6]. This design includes stereo camera for scene capturing, stereo video codec for signal transmission, and autosteroscopic screen for 3D displaying. However, challenges may occur. The first challenge is the resource requirements of stereo video codec. Bundled coding techniques and spatio-temporal prediction in stereo video codec have significant resource requests[5]. The other challenge is 2D terminal users will receive redun- dant views if stereo camera is used. Our recent efforts [7] have found that 3D scene can be reconstructed by content analysis. In the processing of re- construction, depth information of the captured scene is first restored, then the depth information is propagated from key frame to the consequent frames, from which the 3D scene is finally reconstructed. Therefore, to deal with the above challenges, we present a 2D and 3D switchable video com- munication system with name Stereotime, for both tradi- tional and 3D mobile device users. Figure 1 illustrates the framework of our proposed system. In our system, we use the configurations as traditional video communication system, including single camera for scene capturing, video encoder for signal compression, and wireless channel for transmission. The key innovation in the proposed system is a content analysis based 3D scene reconstruction method applied on 3D terminals. In the pro- posed system, scene is captured as 2D video, compressed and transmitted to users but can be displayed in 3D style. For the user with 2D receiver, the system works in a tradi- 473 MM’13, October 21–25, 2013, Barcelona, Spain. ACM 978-1-4503-2404-5/13/10.