Sparse Reconstruction of Compressed Sensing Multi-spectral Data using Cross-Spectral Multi-layered Conditional Random Field Model Edward Li, Mohammad Javad Shafiee, Farnoud Kazemzadeh and Alexander Wong University of Waterloo, 200 University Ave West, Waterloo, Canada, N2L 3G1 ABSTRACT The broadband spectrum contains more information than what the human eye can detect. Spectral information from different wavelengths can provide unique information about the intrinsic properties of an object. Recently compressed sensing imaging systems with low acquisition time have been introduced. To utilize compressed sensing strategies, strong reconstruction algorithms that can reconstruct a signal from sparse observations are required. This work proposes a cross-spectral multi-layerd conditional random field(CS-MCRF) approach for sparse reconstruction of multi-spectral compressive sensing data in multi-spectral stereoscopic vision imaging systems. The CS-MCRF will use information between neighboring spectral bands to better utilize available information for reconstruction. This method was evaluated using simulated compressed sensing multi-spectral imaging data. Results show improvement over existing techniques in preserving spectral fidelity while effectively inferring missing information from sparsely available observations. Keywords: Sparse Reconstruction, Compressive Sensing, Multi-spectral Imaging, Computer vision, Graphical Models 1. INTRODUCTION Having access to information that is available from other wavelengths other than the visible can provide intrinsic properties of the object. Current multispectral (MS) cameras usually utilize filter wheels, liquid-crystal tunable filters or acousto-optical tunable filters. These instruments are expensive, bulky and require very long acquisition time to capture images at multiple wavelengths. Reducing acquisition time and instrument complexity is highly desired for MS imaging. Advances in multi-spectral imaging techniques allow wavelength filtering on the imaging detector at a pixel level. 1–3 This approach can greatly improve acquisition time and ease of use at a cost in spatial resolution as pixels on the detector are assigned to different spectral bands of information. In order to utilize this acquisition strategy, compressed sensing techniques have been proposed. Compressed sensing techniques allow the reconstruction of an entire signal using sparsely yet sufficiently sampled observations. 4 Compressed sensing systems require advanced sparse reconstruction algorithms to infer state information given the observations that are made. 4–6 Reconstruction algorithms that can effectively infer missing information while maintaining spectral fidelity and preserving spatial resolution are strongly desired. Previous work by Kazemzadeh et al. has proposed a multi-layered conditional random field (MCRF) approach for compressively sensed multispectral data. 7 This approach extends CRFs first proposed by Lafferty et al., 8 and models each information band as a MCRF to use spatial and intensity prior information to enhance and infer high spatial resolution states. The MCRF also incorporates an additional layer of abstraction to enforce the quality of the observations into the optimization. One limitation of the MCRF is that additional information from multiple neighboring spectral bands is not utilized. This additional information can be utilized to improve on the reconstruction result. This paper will propose a cross-spectral multi-layered CRF (CS-MCRF) approach for sparse reconstruction of compressed sensing multispectral data. Further author information: (Send correspondence to E.Li) E.Li: E-mail: y245li@uwaterloo.ca, Telephone: 1 657 523 5839