Compression of Multispectral Images: Color (RGB) plus Near-Infrared (NIR) Neda Salamati #1 , Zahra Sadeghipoor #2 , Sabine S¨ usstrunk #3 # School of Computer and Communication Sciences, ´ Ecole Polytechnique F´ ed´ erale de Lausanne (EPFL) Lausanne, Switzerland 1 neda.salamati@epfl.ch 2 zahra.sadeghipoor@epfl.ch 3 sabine.susstrunk@epfl.ch Abstract—We propose a compression framework for four- channel images, composed of color (RGB) and near-infrared (NIR) channels, which exploits the correlation between the visible and the NIR information. The high-frequency components of both visible and NIR scene representations are strongly correlated. By encoding only the DCT components that differ above a chosen threshold, we significantly improve compression ratios for a given quality level. To evaluate our proposed method, we compare our results with standard JPEG compression, as well as PCA-based approaches that are often employed to compress multispectral images. Our experiments show that applying our proposed method yields the same quality at a lower bit-rate, compared to conventional JPEG and PCA-based algorithms. I. I NTRODUCTION Silicon-based camera sensors exhibit significant sensitivity beyond the visible spectrum (400-700 nm). They are able to capture wavelengths up to 1100 nm. Near-infrared (NIR) is the part of the radiation spectrum that ranges from 700 to 1100 nm. Even though this radiation can be captured by silicon, it is usually considered noise and is discarded by fixing a filter (hot-mirror) in front of the sensor. However, retaining instead of eliminating NIR information improves certain tasks in digital photography and computer vision, such as image enhancement [1], scene categorization [2], and illumination estimation [3]. Lu et al. in [4] propose a color filter array (CFA) design that can be employed to simultaneously capture NIR information in addition to red, green, and blue (RGB) channels in the visible part of the spectrum on a single sensor. Compared to conventional color imaging, these emerging applications and this acquisition approach produce larger amounts of data to be transmitted, processed, and stored effi- ciently. Although many multispectral compression algorithms have been proposed [5], [6], [7], [8], to the best of our knowledge, the compression of RGB and NIR has not been specifically addressed so far. Therefore we propose a novel framework for RGB+NIR (RGBN) image compression. There exist several compression algorithms for three- channel RGB images. Currently, one of the vastly employed lossy compression methods is the JPEG standard [9]. This method compresses images by quantizing the Discrete Cosine Transform (DCT) coefficients, and it yields acceptedly good results for visible images. The color images are first trans- formed into the YCbCr color space. The luminance component Fig. 1. Top row: Color (RGB) images. Bottom row: Near-infrared (NIR) images of the same scene. While the image intensities between the two scene representations differ, we can immediately notice that the images are from the same scene due to the similar edge information. (Y) is compressed with a quality better than the chrominance components, as the human visual system is less sensitive to high-frequency loss in chromatic components. In the case of four-channel (RGBN) images, JPEG can be used to compress RGB channels, and it can also be generalized to compress the fourth channel (N) (i.e., NIR as a one-channel gray-scale image is treated like Y in YCbCr and is encoded in the same way). JPEG 2000 [10] differs from JPEG in the transform domain employed. Instead of DCT, multi-level discrete wavelet coeffi- cients are computed for each channel. JPEG 2000 outperforms JPEG in terms of quality at very low bit-rates. Nevertheless, we chose to use JPEG instead of JPEG 2000 for two reasons. First, we are interested mostly in moderate compression, as used in most photographic applications. Second, most cameras still allow only JPEG compression due to the higher computa- tional complexity that JPEG 2000 encoding entails. However, the proposed framework could easily be applied in the wavelet domain. Another approach to compressing RGBN four-channel im- ages is to employ a multispectral compression framework. Pennebaker et al. [11] and Abousleman et al. [7] propose to extend two-dimensional JPEG or JPEG 2000 into a three- dimensional version for multispectral compression. Fowler and Rucker [5] argue against the performance of these compression