FAST DISPARITY ESTIMATION FOR 3DTV APPLICATIONS Yu-Cheng Tseng and Tian-Sheuan Chang Dept. of Electronics Engineering & Institute of Electronics, National Chiao Tung University, Hsinchu, Taiwan ABSTRACT The depth estimation reference software (DERS) algorithm developed by the MPEG 3-D Video Coding could produce high-quality disparity maps for 3DTV applications but suffers from high computational complexity due to its complicated graph-cut optimization. Therefore, this paper proposed a new fast disparity estimation algorithm that could significantly reduce the computational complexity by the downsampled matching cost method. To address the temporal consistency in videos, this paper proposed the no- motion registration for the foreground copy artifact and the still-edge preservation for the flicker artifact. In addition, the occlusion problem is also solved in the proposed algorithm. The experimental results show that our algorithm could generate comparable disparity maps to the DERS algorithm, and only takes 10.8% of its execution time. Index Terms—Disparity estimation, stereoscopic video 1. INTRODUCTION Disparity estimation could find the correspondence among different viewpoint images to generate disparity maps for the view synthesis in 3DTV applications. A disparity estimation algorithm generally consists of the matching cost calculation, cost aggregation, disparity optimization, and disparity refinement. By the general flow, the existing algorithms can be categorized into local, scanline or global approach. The local approach only has the first two steps, so that its complexity is low. The scanline methods are the ones where the optimization is done on each scanline, e.g. dynamic programming. On the other hand, the global approach additionally performs the complicated disparity optimization, such as belief propagation (BP) and graph-cut (GC), to deliver more accurate disparity maps. The final step could refine the disparity map to deal with the fractional disparity, temporal consistency, and occlusion problems. The MPEG 3-D Video Coding (3DVC) has developed the depth estimation reference software (DERS) [1] that can deliver high-quality disparity maps for the 3DTV applications. This algorithm adopts the GC with the segmentation constraint to generate initial disparity map, and then refines it by the plane fitting method. In addition, for the temporal consistency problem, it propagates the previous disparity to current matching costs for the no- motion pixels [2]. If the current disparity is far from previous one, it would suffer from higher temporal cost. However, the DERS algorithm incurs high computational complexity due to the complicated operation of GC and the image segmentation step, even if the acceleration approach [3] is adopted. Moreover, it bears from the foreground copy artifact in the disparity maps because it over-propagates the previous disparity to current frame. Motivated by aforementioned, this paper proposed a fast disparity estimation algorithm for the high computational complexity and the temporal consistency problems. The contributions of this paper are as follows. First, the proposed downsampled matching cost could reduce the computational space without losing disparity precision. Second, the proposed occlusion handling method could accurately detect the occlusion regions and fill them by the window vote to attain better disparity maps. Third, for the temporal consistency problem, the proposed no-motion registration method could address the foreground copy artifact in the DERS algorithm, while the proposed still-edge preservation method could completely avoid the flicker artifact. With the proposed methods, our algorithm could take less computation time than the DERS algorithm and deliver high-quality disparity maps. The rest of this paper is organized as follows. First, Section II presents the key techniques in the proposed high- quality disparity estimation algorithm. Then, Section III compares the proposed algorithm with the DERS algorithm in the execution time and the objective quality evaluation. Finally, Section IV concludes this paper. 2. PROPOSED DISPARITY ESTIMATION ALGORITHM Fig. 1 shows the proposed disparity estimation algorithm that follows the input and output configuration defined in [5] for the 9-view display. In this algorithm, the three view input videos are used to calculate their corresponding disparity maps. In addition, the previous disparity maps and images are fetched to handle the temporal consistency problem. The key techniques in the proposed algorithm are elaborated as follows. 2.1. Downsampled matching cost and disparity map upsampling The disparity estimation suffers from high computational complexity because its computation is dense and proportional to the disparity range, which is related to