On a Generalized Demosaicking Procedure: A Taxonomy of Single-Sensor Imaging Solutions Rastislav Lukac and Konstantinos N. Plataniotis The Edward S. Rogers Sr. Dept. of Electrical and Computer Engineering, University of Toronto, 10 King’s College Road, Toronto, M5S 3G4, Canada {lukacr, kostas}@dsp.utoronto.ca http://www.dsp.utoronto.ca/∼lukacr Abstract. This paper presents a generalized demosaicking procedure suitable for single-sensor imaging devices. By employing an edge-sensing mechanism and a spectral model, the proposed demosaicking framework preserves both the spatial and spectral characteristics of the captured image. Experimental results reported in this paper indicate that the solu- tions designed within the proposed framework produce visually pleasing full color, demosaicked images. 1 Introduction Color filter array (CFA) interpolation or demosaicking is an integral step in single-sensor imaging solutions such as digital cameras, image-enabled wireless phones, and visual sensors for surveillance and automotive applications, [1]-[6]. The CFA is used to separate incoming light into a mosaic of the color components (Fig.1a). The sensor, usually a charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) sensor, is essentially a monochromatic de- vice [1],[7], and thus, the raw data that acquires in conjunction with the CFA con- stitute a K 1 ×K 2 gray-scale image z with scalar pixels z (p,q) , with p =1, 2, ..., K 1 and q =1, 2, ..., K 2 denoting the image row and column, respectively. The two missing color components are estimated from the adjacent pixels using the de- mosaicking process to produce the full-color demosaicked image [8]-[11]. Although a number of CFA have been proposed, the three-color Red-Green- Blue (RGB) Bayer CFA pattern (Fig.1a) [12] is the most commonly used due to the simplicity of the subsequent demosaicking procedure. Assuming the GRGR phase in the first row, a Bayer CFA image z, depicted in Fig.2a, can be trans- formed to a K 1 × K 2 three-channel image x (Fig.2b) as follows [1],[13]: x (p,q) = ⎧ ⎨ ⎩ [z (p,q) , 0, 0] for p odd and q even, [0, 0,z (p,q) ] for p even and q odd, [0,z (p,q) , 0] otherwise. (1) where x (p,q) =[x (p,q)1 ,x (p,q)2 ,x (p,q)3 ] denotes the color vector. The values x (p,q)k indicate the R (k = 1), G (k = 2), or B (k = 3) CFA components. Since the sensor image z is a mosaic-like gray-scale image, the missing components in x (p,q) are set equal to zero to indicate their portion to the coloration of x. V.S. Sunderam et al. (Eds.): ICCS 2005, LNCS 3514, pp. 687–694, 2005. c  Springer-Verlag Berlin Heidelberg 2005