Pattern Recognition 37 (2004) 1219–1231 www.elsevier.com/locate/patcog Eigen-image based compression for the image-based relighting with cascade recursive least squared networks Ze Wang a ; ∗ , Chi-Sing Leung b , Tien-Tsin Wong c , Yi-Sheng Zhu a a Department of Biomedical Engineering, Shanghai Jiao Tong University, HuaShan RD. 1954, Shanghai 200030, China b Department of Electronic Engineering, City University of Hong Kong, Hong Kong c Department of Computer Science Engineering, The Chinese University of Hong Kong, Hong Kong Received 2 December 2002; received in revised form 27 October 2003; accepted 27 October 2003 Abstract This paper presents a principal component analysis (PCA) based data compression method for the image-base relighting (IBL) technology, which needs tremendous reference images to produce high quality rendering. The method contains two main steps, eigen-image based representation and eigen-image compression. We extract eigen-images by the cascade recursive least squared (CRLS) networks based PCA due to the large data dimension. By keeping only a few important eigen-images, which are enough to describe the IBL data set, the data size can be drastically reduced. To further reduce the data size, we use the embedded zero wavelet (EZW) approach to compress those retained eigen-images, and use uniform quantization plus arithmetic coding to compress the representing coecients. Simulation results demonstrate that our approach is superior to that of compressing reference images separately with JPEG or EZW. ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Principal component analysis; Cascade recursive least squared (CRLS); Image-based relighting; Wavelets; Data compression 1. Introduction In the computer graphics literature, image-based mod- elling and rendering (IBMR) [1,2] is proposed as an alternative to the traditional geometry based technology. Its major advantage is that the rendering time complexity is independent of the scene complexity. Therefore, interactive rendering of an arbitrary complex scene becomes possible with the IBMR approach. However, some crucial functions which are trivial in traditional geometry-based technology become no longer obvious when the image-based approach is used. One example is the illumination control. To solve this problem, Wong et al. [3,4] proposed the image-based relighting concept to relight an object (or scene) under a novel illumination condition. The main idea is to synthesize ∗ Corresponding author. Tel.: +86-21-62932830; fax: +86-21- 62933466. E-mail addresses: zwang@sjtu.edu.cn, redhat w@yahoo.com (Z. Wang). a novel image of the scene under a novel lighting condition with some pre-captured reference images under various il- lumination conditions. One reference image can be called an illumination adjustable image (IAI), and the collection of IAI can be considered as an IBL data set. In IBL tech- nology, the quality of image synthesization depends on the number of IAIs used, which means that we have to store a large number of IAIs. Therefore, an ecient compression method is essential to handle the huge IBL data set for real applications. Due to the strong correlations among dierent IAIs, using conventional image compression routines, such as JPEG or wavelet based approaches, to compress each IAI separately is obviously insucient. More complex ap- proaches are needed to fully consider the information redun- dancy in the IBL data set. A powerful feature extraction and data compression tool is principal component analysis (PCA). It can compact most information of the input data set to the rst several principal vectors, where the ith principal component is the eigenvector of the data covariance matrix associated with 0031-3203/$30.00 ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2003.10.009