Af fine Combinations of Adaptive Filters Renato Candido, Magno T. M. Silva, and V´ ıtor H. Nascimento Electronic Systems Engineering Department, Escola Polit´ ecnica, University of S˜ ao Paulo Av. Prof. Luciano Gualberto, trav. 3, n o 158, CEP 05508-900 – S˜ ao Paulo, SP, Brazil E-mails: {renatocan, magno, vitor}@lps.usp.br Abstract—We extend the analysis presented in [1] for the affine combination of two least mean-square (LMS) filters to allow for colored inputs and nonstationary environments. Our theoretical model deals, in a unified way, with any combinations based on the following algorithms: LMS, normalized LMS (NLMS), and recursive-least squares (RLS). Through the analysis, we observe that the affine combination of two algorithms of the same family with close adaptation parameters (step- sizes or forgetting factors) provides a 3 dB gain in relation to its best component filter. We study this behavior in stationary and nonstationary environments. Good agreement between analytical and simulation results is always observed. Furthermore, a simple geometrical interpretation of the affine combination is investigated. A model for the transient and steady-state behavior of two possible algorithms for estimation of the mixing parameter is proposed. The model explains situations in which adaptive combination algorithms may achieve good performance. Index Terms—Adaptive filters, affine combination, steady-state analy- sis, transient analysis, LMS algorithm. I. I NTRODUCTION Recently, an affine combination of two least mean-square (LMS) adaptive filters was proposed and its transient performance analyzed [1]. This method combines linearly the outputs of two LMS filters operating in parallel with different step-sizes. The purpose of the combination is to obtain an adaptive filter with fast convergence and reduced steady-state excess mean-square error (EMSE). Since the mixing parameter is not restricted to the interval [0, 1], this method can be interpreted as a generalization of the convex combination of two LMS filters of [2], [3]. In this paper, we extend the results of [1] by providing a unified analysis, which is valid for colored inputs, nonstationary environ- ments, and combinations based on LMS, NLMS, and RLS algorithms. To explain the behavior of the affine combination of two algorithms, we present a simple geometrical interpretation. Furthermore, we also explain why fast-adaptation of the mixing parameter in general leads to a quite large variance around the optimum value. Then, we find a model for the transient and steady-state behavior of two possible algorithms for estimation of the mixing parameter. In order to simplify the arguments, we assume that all quantities are real. II. PROBLEM FORMULATION A combination of two adaptive filters is depicted in Figure 1. In this scheme, the output of the overall filter is given by y(n)= η(n)y1(n) + [1 - η(n)]y2(n), (1) where η(n) is the mixing parameter, yi (n), i =1, 2 are the outputs of the transversal filters, i.e., yi (n)= u T (n)wi (n - 1), u(n) ∈ R M is the common regressor vector, and wi (n - 1) ∈ R M are the weight vectors of each length-M component filter. We focus on the affine combination of two adaptive algorithms of the following general class wi (n)= wi (n - 1) + ρi (n) Mi (n)u(n)ei (n), i =1, 2, (2) This work is partly supported by CNPq under grants No. 136050/2008-5 and No. 303361/2004-2 and by FAPESP under grants No. 2008/00773-1 and No. 2008/04828-5. where ρi (n) is a step-size, Mi (n) is a symmetric non-singular matrix, ei (n)= d(n) - yi (n) is the estimation error, and d(n) is the desired response. The LMS, NLMS, and RLS algorithms employ the step-sizes ρi (n) and the matrices Mi (n) as in Table I. In this table, μi , ˜ μi and ǫ are positive constants, ‖·‖ is the Euclidian norm, I is the M × M identity matrix, and 0 ≪ λi < 1 is a forgetting factor. For RLS, Mi (n)=  R -1 i (n) is obtained via the matrix inversion lemma [4, Eq.(2.6.4)] applied to  Ri (n), which is an estimate (with forgetting factor λi ) of the autocorrelation matrix of the input signal, i.e., R  E{u(n)u T (n)}, where E{·} is the expectation operator. We assume that d(n) and u(n) are related via a linear regression model, that is, d(n)= u T (n)wo(n - 1) + v(n), where wo(n - 1) is the time-variant optimal solution and v(n) is an i.i.d. (independent and identically distributed) and zero mean random process with variance σ 2 v =E{v 2 (n)}, which plays the role of a disturbance independent of u(n) [4, Sec. 6.2.1]. Furthermore, the sequences {u(n)} and {v(n)} are assumed stationary. In the affine combination, the mixing parameter η(n) is not restricted to the interval [0, 1] and can be adapted via η(n + 1) = η(n)+ μη e(n)[y1(n) - y2(n)], (3) where μη is a step-size, and e(n)= d(n) - y(n) is the estimation error of the overall filter. The recursion (3) was obtained in [1], using a stochastic gradient search to minimize the instantaneous mean- square error (MSE) cost function. In [1], η(n) was constrained to be less than or equal to 1 for all n, to ensure stability of (3). In this paper, we applied this constraint when using (3). The constraint was not necessary when the normalized version of (3) was used (see Sec. V). 1 ( 1 ) n − w 2 ( 1) n − w () yn 1 () y n 2 () y n () n 1- () n () T n u 1 () en 2 () e n () en () dn O ( 1) n − w () vn ( 1) n − w Fig. 1. Affine combination of two transversal adaptive filters. III. STEADY-STATE PERFORMANCE OF ADAPTIVE FILTERS We assume that in a nonstationary environment, the variation in the optimal solution wo follows a random-walk model [4, p. 359], that is, wo(n)= wo(n - 1) + q(n). In this model, q(n) is an i.i.d. vector 236 978-1-4244-2941-7/08/$25.00 ©2008 IEEE Asilomar 2008