Non-linear independent component analysis using series reversion and Weierstrass network P. Gao, W.L. Woo and S.S. Dlay Abstract: The problem of blind separation of independent sources in non-linear mixtures is considered and the focus of this work is on a new type of non-linear mixture in which a linear mixing matrix is sandwiched between two mutually reverse non-linearities. The demixing system culminates to a novel Weierstrass network that is shown to successfully restore the original source signals under the non-linear mixing conditions. The corresponding parameter learning algorithm for the proposed network is presented through formal mathematical derivation. The authors show for the first time a new result based on the theory of forward series and series reversion that is integrated into a neural network to implement the proposed demixer. Simulations, including both synthetic and recorded signals, have been carried out to verify the efficacy of the proposed method. It is shown that the Weierstrass network outperforms other tested independent component analysis (ICA) methods (linear ICA, radial-basis function and multi- layer perceptron network) in terms of speed and accuracy. 1 Introduction During the last decade, tremendous developments have been achieved in independent component analysis (ICA), particularly in array signal processing and signal restoration techniques. The principle aim of ICA is to extract a set of signals as independent as possible from only a set of observations. It is well known that ICA is closely related to blind signal separation (BSS) and many potential exciting applications of ICA have attracted a considerable amount of attent ion in both sci ence and technolog y [1 – 14 and references therein]. However, most existing ICA algorithms focus on linear distortion that may not accord with practical applications [7–14]. In biomedical cases, many physiological signals are non-linearly distorted and thus the identification of non-linear dynamics should be taken into consideration; for example, the auditory nervous system is modelled as a memoryless non-linear system. Another instance is the recording of multiple speech source signals by carbon-button microphones that introduce some form of non-linearity [7, 15, 16]. For non-linear mixing model, linear algorithms fail to extract original signals and become inapplicable because the assumption of linear mixtures is violated and the linear algorithm cannot compensate for the information distorted by the non-linearity. Hence, the search for a non-linear solution becomes urgent and paramount in both theoretical and practical levels. In current literature, non-linear ICA has mostly concen- trated on combining with different kinds of neural networks. In general, these methods can be classified into either gen- erative approaches or signal transformation approaches [13]. In generative approaches, the aim is to find a specific model that represents how the observations are generated and the solution consists of estimating both the source signals and the mixing mapping, whereas signal transform- ation methods construct the separation system and estimate the unknown source signals directly. In both cases, the implementations usually involve the use of neural networks and differ only in terms of cost functions and learning algorithms. Pajunen et al. [17] provided one of the earliest non-linear ICA solutions by using the self-organising maps (SOM). Although theoretically the output of the SOM network can provide the statistically independent vectors, there is no guarantee that the original source signals can be recov- ered by the SOM. As the theoretical foundation of the SOM algorithm is based on rectangular map, the main limit- ation of SOM lies in the inevitable distortion when the source signals differ considerably from the uniform distri- bution. To overcome the disadvantages associated with SOM, Bishop et al. [18] and Pajunen and Karhunen [19] proposed the generative topographic mapping (GTM) approach. However, in order to apply non-uniformly dis- tributed source signals, the GTM method requires the known probability density function (PDF) of the source signals, which may limit the applications of this method. Signal transformation methods based on radial-basis func- tion (RBF) [20] and multilayer perceptron (MLP) [7] neural networks have recently drawn a substantial amount of attention for their flexible non-linear capability. Under the non-linear condition, both methods provide acceptable performance. RBF-based systems can provide fast conver- gence at the cost of less accuracy, whereas MLP can recover the original signals more precisely but suffers from high-computational complexity. Besides the structure of the network, the performance of the demixer also depends on the selection of the non-linear activation function in the hidden neurons. Networks for non-linear ICA such as SOM [17], GTM [18, 19], RBF [20] and MLP with sigmoidal non-linearity [7] are intrinsically non-linear because of the utilisation of fixed non-linearities # IEE, 2006 IEE Proceedings online no. 20045174 doi:10.1049/ip-vis:20045174 Paper first received 16th September 2004 and in revised form 7th June 2005 The authors are with the School of Electrical, Electronic and Computer Engineering, University of Newcastle upon Tyne, UK E-mail: w.l.woo@ncl.ac.uk IEE Proc.-Vis. Image Signal Process., Vol. 153, No. 2, April 2006 115