SPARSE GAUSSIAN NOISY INDEPENDENT COMPONENT ANALYSIS Frosti Palsson, Magnus O. Ulfarsson and Johannes R. Sveinsson Dept. of Electrical Eng., University of Iceland, Reykjavik, ICELAND ABSTRACT There are two main approaches to independent component analysis (ICA); maximization of non-Gaussianity of the sources and the exploitation of temporal correlation in Gaus- sian sources. In this paper, we present a novel sparse noisy ICA model where we have introduced temporal correlation in the sources, described by a first order auto regressive (AR(1)) process. The correlation structure of the sources eliminates the rotational invariance of the estimates, enabling their sep- aration. Using simulated data, we demonstrate both source separation and denoising, where we compare our results to a sparse PCA method and the fastICA method. Additionally, we apply the method on a real hyperspectral dataset. Index Terms— Independent Component Analysis, Spar- sity, Noisy Principal Component Analysis, Source Separa- tion, Denoising 1. INTRODUCTION Principal component analysis (PCA) [1], also known as the Karhunen-Loeve expansion, plays an important role in signal processing, e.g., for exploratory signal analysis and dimen- sionality reduction. PCA decomposes a signal into principal components (PCs) which are orthogonal and ordered accord- ing to their variance. The first PC explains most of the vari- ance of the signal, while the next PC is orthogonal to the first PC and explains second most of the variance of the signal, and so on. For PCA there is an underlying signal processing model [2],[3] called noisy PCA (nPCA). Recent generalizations of nPCA are, e.g., smooth nPCA [4] and sparse variable nPCA (svnPCA) [5]. The main idea of svnPCA is the incorporation of a sparseness vector penalty for automatic variable selec- tion. This is achieved by maximizing a vector ℓ 0 penalized log-likelihood function using the Expectation-Maximization (EM) algorithm [6],[7]. Independent component analysis (ICA) [8] is a technique to separate mixed signals (sources) based on their statisti- cal independence. There are two main approaches to ICA. One is the maximization of the non-Gaussianity of the esti- mated sources and the other approach is to exploit sample de- This work was partly supported by the Research Fund of the University of Iceland and the Icelandic Research Fund (130635-051). pendence in Gaussian sources, i.e., that the samples of each source are correlated, methods that fall into this category are called Gaussian noisy ICA [9]. The main contribution of this paper is a novel sparse Gaussian noisy ICA method, where the sources (PCs) are as- sumed to have temporal correlation described by a first order auto regressive (AR(1)) process. The model is related to the svnPCA model, however, the source estimates in svnPCA are invariant under rotation, making separation impossible. The correlation structure of the sources in the new method elimi- nates the rotation invariance and makes the sources separable. A further generalization of the model is achieved by as- suming that the signal under study is sparse when expressed in a basis such as the orthogonal wavelet basis, which is the choice of basis in this work. We call the new method sparse Gaussian noisy ICA (sgnICA). The proposed method is demonstrated using simulated data and we compare its source separation and denoising performance to the sparse PCA (sPCA) method presented in [10] and the fastICA [11] method. In the sPCA method, the data is transformed to a basis in which the PCs are sparse, using the orthogonal wavelet trans- form. In [10], it is shown that the PCs can be consistently estimated by restricting the PCA to a subset of the variables with variances above a threshold. Instead of adaptively select- ing the threshold, the top k variables are chosen according to their variance. The next step is performing reduced PCA on this subset, retaining the leading r PCs. Finally, the data is reconstructed using the inverse PCA transform and returning to the original basis via the inverse wavelet transform. The organization of the paper is as follows. In Section 2 we derive the sgnICA algorithm. In Section 3 we discuss parameter selection for the proposed method. Section 4 de- scribes the experiments using simulated data and in Section 5 we present denoising of a real hyperspectral remote sensing dataset. Finally, conclusions are drawn in Section 6. 2. THE sgnICA MODEL The sgnICA model is given by y t = Gu t + ǫ t (1) u t = ρu t−1 + η t , t =1, ..., T (2)