Color Stereo Matching Cost Applied to CFA Images Hachem Halawana, Ludovic Macaire, and Fran¸ cois Cabestaing LAGIS, USTL, Bt. P2, Cit Scientifique, 59655 Villeneuve dAscq. France hachem.halawana@ed.univ-lille1.fr, {ludovic.macaire,francois.cabestaing}@univ-lille1.fr http://lagis-vi.univ-lille1.fr/ Abstract. Most color stereovision setups include single-sensor cameras which provide Color Filter Array (CFA) images. In those, a single color component is sampled at each pixel rather than the three required ones (R,G,B). We show that standard demosaicing techniques, used to inter- polate missing components, are not well adapted when the resulting color pixels are matched for estimating image disparities. In order to avoid this problem while exploiting color information, we propose a new matching cost designed for dense stereovision based on pairs of CFA images. Keywords: Color stereovision, CFA image, Demosaicing. 1 Introduction Dense stereo correspondence algorithms are based on measures of the similarity between image locations in a pair of stereo images. Typically, a matching cost is computed at each pixel of the left image for all the shifts in a predefined range, i.e. for a limited set of candidate pixels in the right image. Then, the candidate pixel minimizing the cost is retained and its position yields the disparity. Matching costs assume that homologous pixels have almost the same compo- nent values, but they cope with limited radiometric changes and/or with noise. Common window-based matching costs include the sum of absolute or squared differences (SAD/SSD), normalized cross-correlation (NCC), and census trans- form [6]. Chambon et al. have compared widely used stereo matching costs ap- plied to gray level and color images [2]. They have shown that taking into account color information generally improves the performance of matching costs [8]. Color images can be acquired by two types of cameras: those including three sensors associated with beam splitters and color filters providing the so-called full-color images, and those including a single image sensor. Many recent digital cameras include a single-chip CCD (Charge Coupled Device) or CMOS (Com- plementary Metal Oxide Semiconductor) sensor, to increase image size while reducing device cost. The surface of such a sensor is covered with an array of small spectrally selective filters, arranged in an alternating pattern, so that each photo-sensitive element samples only one of the three color components Red (R), A. Tr´ emeau, R. Schettini, and S. Tominaga (Eds.): CCIW 2009, LNCS 5646, pp. 170–179, 2009. c  Springer-Verlag Berlin Heidelberg 2009