A new approach to corner matching from image sequence using fuzzy similarity index Ambar Dutta a,⇑ , Avijit Kar b , B.N. Chatterji c a Department of Computer Science and Engineering, Birla Institute of Technology, Mesra, Kolkata Campus, Kolkata 700 107, India b Department of Computer Science and Engineering, Jadavpur University, Kolkata 700 032, India c Department of Electronics and Communication Engineering, B. P. Poddar Institute of Management and Technology, Kolkata 700 052, India article info Article history: Received 25 February 2010 Available online 23 December 2010 Communicated by F.Y. Shih Keywords: Corner matching Detection Intensity variation Motion blur Gaussian membership function Fuzzy similarity index abstract Corner matching in image sequences is an important and difficult problem that serves as a building block of several important applications of stereo vision etc. Normally, in area-based corner matching tech- niques, the linear measures like standard cross correlation coefficient, zero-mean (normalized) cross cor- relation coefficient, sum of absolute difference and sum of squared difference are used. Fuzzy logic is a powerful tool to solve many image processing problems because of its ability to deal with ambiguous data. In this paper, we use a similarity measure based on fuzzy correlations in order to establish the cor- ner correspondence between sequence images in the presence of intensity variations and motion blur. The matching approach proposed here needs only to extract one set of corner points as candidates from the left image (first frame), and the positions of which in the right image (second frame) are determined by matching, not by extracting. Experiments conducted with the help of various sequences of images prove the superiority of our algorithm over standard and zero-mean cross correlation as well as one con- temporary work using mutual information as a window similarity measure combined with graph match- ing techniques under non-ideal conditions. Ó 2010 Elsevier B.V. All rights reserved. 1. Introduction Feature matching i.e. determining the correspondences be- tween two sets of features extracted from a pair of views imaging the same scene is a key component in many computer vision appli- cations including discrete motion estimation, object recognition and localization, image registration, camera self-calibration, 3D reconstruction, view synthesis etc. and has been studied exten- sively for decades. There are various interesting features in the dig- ital images, in which corner points are considered as the good candidates, which are image points showing a big 2D-intensity change with a two dimensional structure providing the most valu- able information about image motion. A matching procedure should be followed after the extraction of any local features from an object in order to identify and locate the object. When feature points are detected for the purpose of matching, the key property of the detector is repeatability (or consistency): in different views of the same scene, the detector should extract the same points, de- spite the variations due to a change in perspective or lighting con- ditions. We considered corners as the local features in this paper. The problem is how to automatically estimate image feature correspondences between two or more images, while at the same time not assigning matches incorrectly. A number of approaches have been proposed to address the the- oretical and applied issues of correspondence problem of which two approaches are more popular. One is based on some similar- ity/dissimilarity measures discussed in Section 2, and the other uses the feature point properties. One possible strategy is to re- quire that the corners in a pair have similar shapes. A corner shape is defined as a small area around the feature point, belonging to the same scene object as this feature point. A method to extract the corner from its background is therefore required. Once corner shapes have been extracted, the Hamming distance between the obtained binary foreground/background maps is computed and pairs for which this distance is above some threshold are eliminated. Some matching approaches proposed must extract independently two sets of candidates from two consecutive frames, respectively. Obviously, the total time for matching process includes the cost of extracting two sets of candidates. The match- ing approach proposed by us needs only to extract one set of corner points as candidates from the first frame, and the positions of which in the second frame are determined by matching, not by extracting. In the matching process, the cross correlation algo- rithms are applied to calculate the similarity of two small regions, one of which is the template window surrounding the gray value corner in frame-1 and the other is searched in frame-2 for 0167-8655/$ - see front matter Ó 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2010.12.006 ⇑ Corresponding author. Fax: +91 33 24414299. E-mail addresses: adutta@bitmesra.ac.in (A. Dutta), avijit_kar@cse.jdvu.ac.in (A. Kar), bnchatterji@gmail.com (B.N. Chatterji). Pattern Recognition Letters 32 (2011) 712–720 Contents lists available at ScienceDirect Pattern Recognition Letters journal homepage: www.elsevier.com/locate/patrec