J Sign Process Syst DOI 10.1007/s11265-014-0948-2 Selection of a Closed-Form Expression Polynomial Orthogonal Basis for Robust Nonlinear System Identification Mariem Kallel Smaoui · Yousra Ben Jemˆ aa · M´ eriem Jaidane Received: 19 June 2013 / Revised: 19 June 2014 / Accepted: 21 August 2014 © Springer Science+Business Media New York 2014 Abstract Polynomial nonlinear system identification suf- fers from numerical instability related to the ill-conditioning of the involved matrices. Orthogonal methods consist in conditioning the input signal in order to reduce the eigen- values spread of the correlation matrix. The selection of an appropriate orthogonalization procedure, for robustness improvement, resort to signal statistics. Consequently, sev- eral orthogonal polynomial bases are proposed in the lit- erature. Most of them use iterative processes and imply a considerable computational cost. Actually, with the growth of real-time applications, it is important to generate non- iterative orthogonal procedures, allowing optimization of algorithm-architecture adequacy. Our paper’s motivation is based on the complexity reduction aspect related to the orthogonalization process. Therefore we propose to focus on closed-form expressions of commonly used orthogonal polynomials bases namely the Shifted-Legendre orthogo- nal polynomials and the Hermite polynomials. A robustness study in terms of numerical stability enhancement of the two bases is carried on. Through comparative simulations results, the basis allowing the best matrix conditioning and an ease generalization for real-time applications with less restrictive hypothesis is selected in order to study the robustness of a polynomial nonlinear system identification M. K. Smaoui · M. Jaidane Signals and Systems research Unit, National Engineering School of Tunis, BP 37, Le Belv´ ed` ere, 1002, Tunisia e-mail: U2S@enit.rnu.tn Y. Ben Jemˆ aa () Department of Informatics and Applied Mathematics, National Engineering School of Sfax, Sfax 3038, Tunisia e-mail: yousra.benjemaa@enis.rnu.tn scheme. Computer simulations are carried out to empha- size the advantages of the proposed scheme using different performance criteria for both the optimal case and the adap- tive case, in terms of numerical stability and convergence rate. We propose to experiment the identification of the power amplifiers, for radio mobile applications and the loudspeakers for audio applications. Keywords Nonlinear system · Polynomial identification · Orthogonalization procedures · Robustness · Numerical stability · Complexity reduction 1 Introduction With the expansion of nonlinear applications such as mul- timedia transmission, wireless and satellite communication, nonlinear device characterization is of growing interest. They have contributed to the identification and compensa- tion of communication channel distortions. Nonlinear filtering has been developed for both the opti- mal and the adaptive context. Several models are generated to take into account the nonlinearities in communication systems, among other examples, the Volterra series [1–3], multilayer perceptron neural networks [4], radial basis function network [5] and memoryless polynomials [6, 7]. First proposed by Wiener, the Volterra series are essen- tially dedicated to systems with fading memories and have been extensively developed for adaptive filtering. How- ever, they have shown several notable limitations because of the huge number of parameters characterizing the Volterra kernels [5]. New approaches have been proposed for signif- icantly reducing the parametric complexity of the Volterra series [8].