Please cite this article in press as: A.S. Malik, et al., A Fuzzy-Neural approach for estimation of depth map using focus, Appl. Soft Comput. J. (2010), doi:10.1016/j.asoc.2010.05.030 ARTICLE IN PRESS G Model ASOC-901; No. of Pages 14 Applied Soft Computing xxx (2010) xxx–xxx Contents lists available at ScienceDirect Applied Soft Computing journal homepage: www.elsevier.com/locate/asoc A Fuzzy-Neural approach for estimation of depth map using focus Aamir Saeed Malik a,∗ , Humaira Nisar b , Tae-Sun Choi b a Department of Electrical & Electronic Engineering, Universiti Teknologi Petronas, Bandar Sei Iskandar, 31750 Tronoh, Perak, Malaysia b Department of Mechatronics, Gwangju Institute of Science and Technology, Republic of Korea article info Article history: Received 1 October 2009 Received in revised form 5 May 2010 Accepted 30 May 2010 Available online xxx Keywords: Depth map estimation 3D shape recovery Fuzzy logic Neural Network Back propagation abstract Depth map is used for recovery of three-dimensional structure of the object which is required in many high level vision applications. In this paper, we present a new algorithm for the estimation of depth map for three-dimensional shape recovery. This algorithm is based on Fuzzy-Neural approach using shape from focus (SFF). A Fuzzy Inference System (FIS) is designed for the calculation of the depth map and an initial set of membership functions and fuzzy rules are proposed. Then Neural Network is used to train the FIS. The training is done using back propagation algorithm in combination with the least squares method. Hence, a new set of input membership functions are generated while discarding the initial ones. Lastly, the trained FIS is used to obtain final depth map. The results are compared with five other methods including the traditional SFF method and the Focused Image Surface SFF method (FISM). Six different types of objects are used for testing the proposed algorithm. © 2010 Elsevier B.V. All rights reserved. 1. Introduction Depth estimation is an important parameter for robot vision, 3D scene modeling, motion, etc. using single or multiple cameras. Various passive techniques have been proposed in the literature to estimate depth including depth from focus, defocus, stereo, etc. [1,2]. We study depth/shape from focus. Under suitable conditions, different three-dimensional (3D) scenes generate different accom- modation cues, and therefore it is possible to distinguish them from their images. Focus is an accommodation cue that can be measured from blurring in the image, which increases with the dis- tance of imaging system from the plane of focus. Techniques that retrieve spatial information, by looking at multiple images of the same scene, taken with different geometries or position of imaging devices, are classified as shape from focus (SFF) or depth from focus [3,4]. The objective of shape from focus (SFF) is to find out the depth of every point of the object from the camera lens. The shape from focus method is based on obtaining a series of images for an object by a single camera at different distances between lens and object. Then by using a quality measure of focus for each point, the image frame for which that point is well-focused is calculated. The basic image formation geometry is shown in Fig. 1. In the figure, the parameters related to the camera are already known. We need to calculate ‘u’, i.e., depth of object from the lens. We make a depth map by calculating ‘u’ for every pixel. We can use ∗ Corresponding author. E-mail address: aamir saeed@petronas.com.my (A.S. Malik). the lens formula to calculate ‘u’. If the image detector (ID) is placed exactly at a distance v, sharp image P ′ of the point P is formed at v (see Fig. 1) while P ′′ is the blurred circle (defocused points). Then the relationship between the object distance u, focal distance of the lens f, and the image distance v is given by the Gaussian lens law: 1 f = 1 u + 1 v Given the position of the focused image for each point (image frame for which a point is well-focused), its position in the scene is uniquely determined by using the lens formula. To measure the true focus point requires large number of images with incremental distance moved towards focus plane. To detect the true focus point from finite number of images, various focus measures have been proposed by researchers. A focus measure is a quantity which measures the degree of blurring of an image; its value is a maximum when the image is best focused and decreases as blurring increases. Fig. 2 shows a focus measure curve for a point in the image calculated using Sum of Modified Laplacian (SML) (camera parameters: focal length = 35 mm, aperture diam- eter = 35/4 mm, F-number = 4). In Fig. 2, the peak area shows the pixel being focused in those frames while being defocused in other frames. However, as we move away (on both sides of the peak), the value decreases indicating increase in blur. Depth estimation, like all aspects of computer vision, has some inherent error. Some sources of this error are noise in the image, imprecision in computations, ambiguity in interpreting depth information, camera lens aberrations, etc. These errors combine to create an uncertainty and error in the result. Therefore, various 1568-4946/$ – see front matter © 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.asoc.2010.05.030