70 P. KAMENCAY, M. BREZNAN, R. JARINA, P. LUKAC, M. ZACHARIASOVA, IMPROVED DEPTH MAP ESTIMATION … Improved Depth Map Estimation from Stereo Images Based on Hybrid Method Patrik KAMENCAY, Martin BREZNAN, Roman JARINA, Peter LUKAC, Martina ZACHARIASOVA 1 Dept. of Telecommunications and Multimedia, University of Zilina, Univerzitna 8215/1, 010 26 Zilina, Slovakia patrik.kamencay@fel.uniza.sk, martin.breznan@fel.uniza.sk, roman.jarina@fel.uniza.sk, peter.lukac@fel.uniza.sk, martina.zachariasova@fel.uniza.sk Abstract. In this paper, a stereo matching algorithm based on image segments is presented. We propose the hybrid segmentation algorithm that is based on a combination of the Belief Propagation and Mean Shift algorithms with aim to refine the disparity and depth map by using a stereo pair of images. This algorithm utilizes image filtering and modified SAD (Sum of Absolute Differences) stereo matching method. Firstly, a color based segmentation method is applied for segmenting the left image of the input stereo pair (reference image) into regions. The aim of the segmentation is to simplify representation of the image into the form that is easier to analyze and is able to locate objects in images. Secondly, results of the segmentation are used as an input of the local window-based matching method to determine the disparity estimate of each image pixel. The obtained experimental results demonstrate that the final depth map can be obtained by application of seg- ment disparities to the original images. Experimental re- sults with the stereo testing images show that our proposed Hybrid algorithm HSAD gives a good performance. Keywords Image segmentation, disparity, Mean Shift, Belief propagation, SAD, HSAD, depth map, 3D image, stereo matching. 1. Introduction Recovery of 3D shape is a critical problem in many vision application domains such as object modeling, scene understanding and high level visual activity recognition or robotics applications [3]. Obtaining a precise and accurate depth map is the ultimate goal for 3D shape recovery and 3D image reconstruction. The topic of the paper is focused on the process of the depth map computation from the images that are captured by the cameras placed in such positions so that a scene is taken from two slightly different views (angles). By using modern stereo vision systems, we can accurately estimate the depth. The Bumblebee stereo vision camera system from Point Grey Research is a two lens camera and forms the basis of our research system [13]. This camera produces a disparity map in real time. A simple procedure for depth map generation from the stereo camera system of Point Grey is shown in Fig. 2. Fig. 1. The Bumblebee stereo camera system by Point Grey Research [13]. Disparity map computation is one of the key problems in 3D computer vision. A large number of algorithms have been proposed to solve this problem. However, since the problem is an ill-posed, a satisfying solution has not been found yet [1], [2]. An assignment of the stereo matching algorithm is to find such points in the both images that represent the same scene point. Fig. 2. System platform overview [18]. For the epipolar rectified image pair, each point in the left image lies on the same horizontal line (epipolar line) as in the right image. This approach is used to reduce a search space for depth map computation algorithms. The depth of an image pixel is the distance of the corresponding space point from the camera center. To estimate the depth map and detect 3D objects, the corresponding pixels in the left and right images have to be matched. The proposed system for depth recovery starts with acquisition of images, which are calibrated and rectified. This algorithm consists of the following stages: