Theoretical analysis of correlation-based quality measures for weighted averaging image fusion Chuanming Wei and Rick S. Blum Abstract-Recently introduced correlation-based quality mea- sures have received lots of attention due to the fact that they do not need ground-truth reference images to evaluate the performance of image fusion algorithms. In this paper we focus on theoretical analysis of these correlation-based quality measures when they are used to judge the performance of weighted averaging image fusion algorithms. The purpose of this paper is to rigorously prove that the correlation-based quality measures have some undesired behavior under certain conditions. We employ a statistical model for the observed sensor images and study the properties of these correlation-based quality measures. Our analysis shows that when we change the power of the desired signal or the noise in the input images, these correlation- based quality measures exhibit bad behaviors in some cases, indicating higher quality when lower quality is evident. The sufficient conditions for when the undesired behaviors occur and the intuitive explanation for our observation are given in this paper. Investigations with real images also demonstrate the utility of the theoretical analysis, by illustrating its predictive capabilities. Index Terms-Image fusion, correlation-based quality mea- sure, weighted averaging, mutual information. I. INTRODUCTION The purpose of image fusion is to produce a single image which contains more information about a scene than any of the individual source images [1]. The different input images to be fused may come from either one sensor monitored over a pe- riod of time or from different sensors. The fused image should be more helpful for human visual or machine perception. Image fusion is widely used in many application areas like concealed weapon detection (CWD), remote sensing, medical diagnosis and video surveillance, etc [2]. Recently, several image fusion quality measures have been introduced [3]-[5] which are all based on a similar philosophy. The philosophy involves considering correlations between the images input into the fusion algorithm and the image produced by the fusion algorithm, possibly after some preprocessing. The correlation is intended to provide a measure of the amount of information transferred from the input images to the fused The authors are with the Electrical and Computer Engineering Depart- ment, Lehigh University, Bethlehem, PA 18015, e-mail: chw207@lehigh.edu, rblum@ece.lehigh.edu. Research was sponsored by the Army Research Laboratory and was accom- plished under Cooperative Agreement Number W911NF-06-2-0020, and by a grant from the Commonwealth of Pennsylvania, Department of Community and Economic Development. The views and conclusions contained in this document are those of the authors and should not be interpreted as the official policies, either express or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon. image. Higher correlation implies more information transfer and thus is considered to result in better fused images. Here we shall refer to the methods from [3]-[5] as correlation-based quality measures. The correlation-based quality measures have received a lot of attention in the past few years because they don't require a ground-truth reference image which is usually difficult to obtain in most applications. Here we focus on three correlation metrics which are the mutual information quality measure [3] [6], the universal image quality index (UIQI) [4] and the edge-based correlation metric [7] [5]. In [6], the authors employed a statistical model [8] which approximates the observed sensor images as an additive sum of the scene of interest plus the random distortion part, which we call noise for simplicity. The authors of [6] studied the mutual information-based quality measure from [3]. In particular they studied how this quality measure reacted to increases in the noise power of the input images for weighted averaging image fusion algorithms. They made an interesting observation. In some cases the quality measure indicated higher quality for increased input noise power. In this paper, we extend the study in [6] by performing theoretical analysis for three different correlation-based quality measures applied to the weighted averaging fusion algorithm. We study the closed-form expres- sions for quality by employing the image formation model described in [8] and analyze quality changes with respect to changes in noise power. Our analysis shows that all of these three correlation-based quality measures sometimes behave opposite to what is expected. Sufficient conditions for the unexpected behavior along with an intuitive explanation are provided in this paper. The paper is organized as follow. Section II presents the statistical model for the observed sensor images. Our main findings are stated in Section III. In Section IV we de- scribe three correlation-based quality measures and derive their closed-form expressions. An intuitive explanation for our findings is provided in Section V. Then, Section VI provides some numerical results. Concluding remarks are given in Section VII. II. IMAGE FORMATION MODEL To perform analytical studies, it is typically necessary to develop a model for the process under consideration. Here we employ a very simple model which was first proposed in [8]. We hope that other researchers will build on these ideas and propose more comprehensive models and perform more comprehensive studies based on these models. 978-1-4244-2734-5/09/$25.00 ©2009 IEEE 363