DISPARITY DEPENDENT SEGMENTATION BASED STEREO IMAGE CODING Rahul Shukla † and Hayder Radha § † Audio-Visual Communications Laboratory Swiss Federal Institute of Technology Lausanne (EPFL), Switzerland § Department of ECE, Michigan State University, East Lansing, USA email: rahul.shukla@epfl.ch, radha@egr.msu.edu ABSTRACT In this paper, we propose a novel rate-distortion (R-D) op- timized disparity based coding scheme for stereo images. This new scheme efficiently integrates the coding of the disparity field with the residual image obtained via dis- parity estimation/compensation (DE/DC) in an R-D frame- work. The scheme first performs a quadtree decomposi- tion of the target image and computes the disparity infor- mation along with the residual image for each node in the tree. An R-D based algorithm is then used for optimum bit allocation among the different quadtree nodes. The pro- posed scheme further jointly encodes the neighboring nodes with similar disparity information to attain higher cod- ing gains. We present simulation results for the proposed scheme and compare these results with the performance of a fixed block size DE/DC based JPEG2000 stereo image coder. Our simulations show that the proposed scheme out- performs the fixed block size based disparity compensated JPEG2000 coder by more than 0.5 dB. 1. INTRODUCTION It is well known that the binocular or disparity information extracted from stereo images plays a crucial role in sev- eral fields like computer vision, remote sensing/handling, tele-presence style video conferencing, tele-medicine and 3-D cinema. In particular, the majority of multi-view vi- sual coding schemes are based on disparity-field estima- tion and compensation. Moreover, the recent increase in stereo visual applications translates into a growing demand for efficient methods for the transmission and storage of stereo image/video pairs. If the stereo images are com- pressed and transmitted independently using the standard coding schemes, then the required bandwidth would need to be doubled. However, due to the binocular dependency between the stereo images, they contain significant redun- dant information in the form of inter-frame redundancy [7]. This work was supported by the Swiss National Science Foundation grant 20-63664.00. Thus, by exploiting this binocular redundancy in addition to the intra-frame redundancy present in the two constituent images, we can achieve significant data compression with- out sacrificing the overall image quality. Coding of stereoscopic images is generally based on ex- ploiting the correlation between the left and right images. This is achieved by computing a disparity field between the stereo image pair [5, 7, 8]. The disparity field represents the amount of shift one needs to perform on the pixels within one image (target) to find the corresponding pixels in the other image (reference). A popular approach for computing the disparity field is to partition the target image into a set of blocks and perform a block matching algorithm to find the best match in the reference image. For coding applications, this approach is known as disparity estimation/disparity compensation (DE/DC) due to its resemblance to motion estimation/motion compensation (ME/MC) methods, which are popular for video coding. The key concept of stereo im- age compression based on DE/DC is to use one of the im- ages in the stereo pair as a reference and to predict the other image (the target). Disparity compensation based prediction for stereo im- ages was first introduced by Lukacs [5]. In [7], Perkins pro- posed the conditional coder/decoder structure for the stereo image coding and analyzed the R-D behavior of this struc- ture. Aydinoglu et al. presented a coding scheme which combines disparity compensation and transform coding of the residual image in the single framework using an adap- tive incomplete transform [1]. Hierarchical block matching algorithms have been used to generate a more homogeneous disparity fields that lead to better coding efficiency [10, 15]. In [2], a novel scheme is presented for the efficient ex- ploitation of the zerotree algorithm in stereo image coding applications. Disparity field estimation based on low level features, such as edges, and on model/object based meth- ods have also been used [13]. In [15], feature/object based DE/DC scheme is proposed. This scheme determines a set of objects/features in both images and seeks correspondence between the two sets. Jiang [3] proposed a hybrid approach