Compressive Demosaicing Abdolreza Abdolhosseini Moghadam #1 , Mohammad Aghagolzadeh #2 , Mrityunjay Kumar ∗3 , and Hayder Radha #4 # Dept. of Electrical & Computer Eng., Michigan State University, ∗ Eastman Kodak Company, Rochester, NY, USA. 1 abdolhos@msu.edu, 2 aghagol1@msu.edu , 3 mrityunjay.kumar@kodak.com, 4 radha@msu.edu Abstract—A typical consumer digital camera uses a Color Filter Array (CFA) to sense only one color component per image pixel. The original three-color image is reconstructed by interpolating the missing color components. This interpolation process (known as demosaicing) corresponds to solving an under- determined system of linear equations. In this paper, we show that by replacing the traditional CFA with a random panchromatic CFA, recent results in the emerging field of Compressed Sensing (CS) can be used to solve the demosaicing problem in a novel way. Specifically, during the image reconstruction process, we exploit the fact that the multi-dimensional color of each pixel has a compressible representation in a (possibly overcomplete) color system. While adhering to the “single color per pixel sensing” constraint at the sensing stage, during the reconstruction process we utilize the inter-pixel correlation by exploiting the compress- ible representation of the overall image in some sparsifying bases. Depending on the CFA, sparsifying bases and the color system, we form an underdetermined system of linear equations and find the sparsest solution for the color image by utilizing a CS solver. We illustrate that, for natural images, the proposed Compressive Demosaicing (CD) framework visually outperforms leading demosaicing methods in a consistent manner; in many cases it achieves clear visible improvements in a significant way. I. I NTRODUCTION Motivated by cost constraints, most low-cost consumer grade digital camera systems are currently designed to (a) sense only one color component per image pixel and (b) interpolate the other missing color components (at each pixel) during reconstruction. The sensing process, which employs a Color Filter Array (CFA), maps each pixel to a single color based on a color pattern. The CFA color pattern and the interpolation process (widely known as demosaicing) have a significant impact on the quality of the reconstructed image. The most popular CFA pattern is the Bayer color pattern that employs two green filters, one red, and one blue filter in each 2 × 2 block within the CFA. Many other CFA patterns have been proposed including ones that are based on secondary colors [3]. There has been a great deal of attention paid to the demosaicing problem, and consequently, a flurry of algorithms has been proposed [2]-[11]. Several recent papers on image demosaicing provide an excellent overview of leading approaches and their classification (e.g., spatial versus frequency domains) [1]. In general, demosaicing algorithms exploit the correlation that exists among adjacent MMSP’10, October 4-6, 2010, Saint-Malo, France. 978-1-4244-8112- 5/10/$26.00 2010 IEEE pixels (inter-pixel correlation) and among color planes (inter- channel correlation) [1]-[11]. Meanwhile, the area of Compressed Sensing (CS) [12] has attracted a great deal of attention recently. The problem of CS targets the sparsest solution of an underdetermined system of linear equations. Similarly, the problem of demosaicing is basically an attempt to finding a solution to an underde- termined system of linear equations where for each pixel, one linear sample of three color components is sensed. In principle, CFA-based image capture represents a three-to-one compressed sensing. Hence, utilizing the rich results developed in the CS area to solve the demosaicing problem seems plausible. In this paper, we present Compressive Demosaicing (CD), a framework to demosaic natural images by employing aspects from the theory of CS. More specifically, instead of finding the missing color components of a pixel, we find an equivalent compressible description of the same image. This equivalent description of the image is essentially the redundant representation of that image with minimal inter-channel and inter-pixel correlations. In words, given the CFA samples, the proposed CD framework finds the transform coefficients of the image (with respect to a sparsifying frame or basis) in a redundant color space, by algorithms developed in the CS area to reconstruct the three-color image. We employ a random panchromatic CFA during the sensing stage of our proposed framework. It is important to highlight that the proposed compressive demaosaicing framework differs significantly from other re- cent attempts for combining CS and CFA sensing. In par- ticular, the utility of CS for sensing color images has been proposed in [19]. Our proposed CD framework departs from prior work in many ways both in terms of the problem objectives and also the approach to solve that problem. For instance, [19] requires a CS-camera [18] (where for each pixel, a linear measurement of the whole image is sensed) and hence requires drastic changes in the design of digital cameras which might not be feasible (at least at the present time). On the other hand, in our method, for each pixel, we only sense a linear combination of color components of that (single) pixel, which can be achieved simply by employing a random panchromatic CFA. Hence, we strictly adhere to the “single color per pixel” constraint. Second, [19] utilizes a joint sparsity model to recover a sparse representation of the color image. On the other hand, we utilize a novel combination of Equiangular Tight Frames (ETFs) along with YUV color system to de-correlate the color components of an image. 978-1-4244-8112-5/10/$26.00 ©2010 IEEE 105