Spatially Adaptive Color Filter Array Interpolation for Noiseless and Noisy Data Dmitriy Paliy, 1 Vladimir Katkovnik, 1 Radu Bilcu, 2 Sakari Alenius, 2 Karen Egiazarian 1 1 Institute of SignalProcessing, TampereUniversity of Technology, P.O. Box 553, 33101 Tampere, Finland 2 Nokia Research Center, Tampere, Finland Received 25 January 2007; accepted 29 August 2007 ABSTRACT: Conventional single-chip digital cameras use color filter arrays (CFA) to sample different spectral components. Demosaicing algorithms interpolate these data to complete red, green, and blue values for each image pixel, to produce an RGB image. In this article, we propose a novel demosaicing algorithm for the Bayer CFA. For the algorithm design, we assume that, following the concept proposed in (Zhang and Wu, IEEE Trans Image Process 14 (2005), 2167–2178), the initial interpolation estimates of color channels contain two addi- tive components: the true values of color intensities and the errors that are considered as an additive noise. A specially designed signal- adaptive filter is used to remove this so-called demosaicing noise. This filter is based on the local polynomial approximation (LPA) and the paradigm of the intersection of confidence intervals applied to select varying scales of LPA. This technique is nonlinear and spa- tially-adaptive with respect to the smoothness and irregularities of the image. The presented CFA interpolation (CFAI) technique takes signif- icant advantage from assuming that the original data is noise-free. Nevertheless, in many applications, the observed data is noisy, where the noise is treated as an important intrinsic degradation of the data. We develop an adaptation of the proposed CFAI for noisy data, inte- grating the denoising and CFAI into a single procedure. It is assumed that the data is given according to the Bayer pattern and corrupted by signal-dependant noise common for charge-coupled device and complementary-symmetry/metal-oxide semiconductor sensors. The efficiency of the proposed approach is demonstrated by experimental results with simulated and real data. V V C 2007 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 17, 105–122, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.20109 Key words: Bayer pattern; color filter array interpolation; spatially adaptive denoising; sensor noise I. INTRODUCTION The common approach in single-chip digital cameras is to use a color filter arrays (CFA) to sample different spectral components like red, green, and blue. The sensor records one value per pixel location. The resulting image is a gray-scale mosaic-like one. Demosaicing algorithm interpolates sets of complete red, green, and blue values for each pixel, to make an RGB image. Independent interpolation of color channels usually leads to drastic color distor- tions. The way to effectively produce a joint color interpolation plays a crucial role for demosaicing. Modern efficient algorithms exploit several main facts. The first is the high correlation between the red, green, and blue channels for natural images. As a result, all three color channels are very likely to have the same texture and edge locations. The second fact is that digital cameras use the CFA in which the luminance (green) chan- nel is sampled at the higher rate than the chrominance (red and blue) channels. Therefore, the green channel is less likely to be aliased, and details are preserved better in the green channel than in the red and blue channels (Gunturk et al., 2002). Also, the CFA is a crucial element in design of single-sensor digital cameras. Different characteristics in design of CFA affect both performance and com- putational efficiency of the demosaicking solution (Adams et al., 1998; Lukac, 2005a). The fundamentals about digital color image acquisition with single-sensor can be found in (Parulski and Spauld- ing, 2002; Lukac and Plataniotis, 2006b). Considering the fact that the Bayer CFA (Bayer, 1976) (see Fig. 1) is one of the most often exploited today, in this article, we focus on techniques for this particular CFA. A. Correlation Models. There are two basic interplane correla- tion models: the color difference rule (Laroche and Prescott, 1994; Hamilton and Adams, 1997) and the color ratio rule (Kimmel, 1999; Lukac et al., 2004a). The first model asserts that intensity dif- ferences between red, green, and blue channels are slowly varying, that is the differences between color channels are locally nearly- constant (Laroche and Prescott, 1994; Hamilton and Adams, 1997; Adams, 1998; Lukac and Plataniotis, 2004a, 2005b; Hirakawa and Parks, 2005a; Li, 2005; Zhang and Wu, 2005). Thus, they contain low-frequency components only, making the interpolation using the color differences easier (Hirakawa and Parks, 2005a). Correspondence to: Dmitriy Paliy; e-mail: dmitriy.paliy@tut.fi This work was supported by the Finnish Funding Agency for Technology and Innovation (Tekes), AVIPA Project. ' 2007 Wiley Periodicals, Inc.