Neurocomputing 49 (2002) 25–38 www.elsevier.com/locate/neucom Blind source separation using Renyi’s -marginal entropies Deniz Erdogmus ∗ , Kenneth E. Hild II, Jose C. Principe Computational NeuroEngineering Laboratory, University of Florida, Gainesville, FL 32611, USA Received 13 March 2001; accepted 22 October 2001 Abstract We have recently suggested the minimization of a nonparametric estimator of Renyi’s mutual information as a criterion for blind source separation. Using a two-stage topology, consisting of spatial whitening and a series of Givens rotations, the cost function reduces to the sum of marginal entropies, just like in the Shannon’s entropy case. Since we use a Parzen window density estimator and eliminate the joint entropy by employing an orthonormal demixing matrix, the problems of probability density function inaccuracy due to truncation of series expansion and the estimation of joint pdfs in high-dimensional spaces (given the typical paucity of data) are avoided, respectively. In our previous formulation, the algorithm was restricted to Renyi’s second-order entropy and Gaussian kernels for the Parzen window estimator. The present work extends the previous results by formulating a new estimation methodology for Renyi’s entropy, which allows the designer to choose any order of entropy and any suitable kernel function. Simulations illustrate that the proposed method compares favorably to Hyvarinen’s FastICA, Bell and Sejnowski’s Infomax and Common’s minimum of mutual information. c  2002 Elsevier Science B.V. All rights reserved. Keywords: Blind source separation; Renyi’s entropy; Parzen estimation 1. Introduction A typical blind source separation (BSS) system operates on observations that are obtained by passing unknown independent signals through an unknown mixing matrix. The block diagram of the BSS scheme we assume is given in Fig. 1. An observation vector z = H T s is obtained from the sources. First, a spatial whitening is applied to the observed data, x = W T z, where the whitening transform W is evaluated from the * Corresponding author. E-mail address: deniz@cnel.u.edu (D. Erdogmus). 0925-2312/02/$-see front matter c  2002 Elsevier Science B.V. All rights reserved. PII:S0925-2312(02)00526-X