Multi-objective optimisation technique to design digital filters for modulated multi-rate systems F. Cruz-Rolda ´n, C. Heneghan, J.B. Sa ´ez-Landete, M. Blanco-Velasco and P. Amo-Lo ´pez An improved technique to design prototype filters for nearly-perfect reconstruction modulated filter banks and transmultiplexers is shown. This technique is based on the frequency sampling approach, and the formulation is presented as a multi-objective optimisation problem. Results indicate that the new formulation provides arbitrary-length prototype filters with even better quality parameters. Introduction: Within the general class of M-channel filter banks, the class of nearly-perfect reconstruction (N-PR) modulated filter banks (MFBs) has been widely studied and used in digital signal processing for applications in compression and in multicarrier-based digital trans- mission. The main advantage of NPR against perfect reconstruction (PR) systems relies on higher selectivity and discrimination, and there- fore, both spectral separation between subbands (or subchannels) and interference rejection is provided. Moreover, MFBs offer theoretical and practical advantages, such as the design process concentrates on optimising only one prototype filter [1]. This Letter focuses on this topic, showing an improved technique to design lowpass linear-phase prototype filters based on a multi-objective optimisation problem, with a search space limited to a small number of variables. The resulting linear-phase prototype filters have a huge number of transmission zeros and are valid to design M-channel maximally decimated cosine modulated filter banks (CMFBs), 2M-channel modified-discrete Fourier transform (MDFT) FBs [1, 2], and the corresponding transmultiplexers (TMUXs). As a result, the resulting system performs significantly better than other approaches developed using similar techniques. Proposed technique: The design of a prototype lowpass filter which has bandwidth approximately p/M is presented. The passband should have unit magnitude, with minimal ripple, and the stopband should be zero valued. The transition band should be as narrow as possible. The optimisation procedure to design such a zero-phase filter p[n] using a frequency-sampling technique is as follows. 1. Select the desired filter length N. 2. Select the required number L of samples in the transition band. Let v p ¼ i p (2p)/N and v s ¼ (i p þ L þ 1)(2p)/N be the frequencies corre- sponding to the last sample of the passband and the first sample of the stopband, respectively. The transition band centre must be initially located as close as possible to the frequency v c ¼ p/(2M ), with v s ’ n/M. 3. Initialise the N samples of the prototype filter magnitude response as P T ¼ P½0 P½1  P½N  1   ð1Þ where each element of the above vector is obtained as P T   q ¼ 1  1 |fflfflfflfflffl ffl{zfflfflfflfflffl ffl} i p samples ðpassbandÞ lði p þ 1Þ  lði p þ LÞ |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl} Lsamples ðtransitionbandÞ 0  q  bðN  1Þ=2c 0  0 |fflfflfflfflfflfflffl{zfflfflfflfflfflffl} R  L  i p samples ðstopbandÞ 2 6 6 6 4 3 7 7 7 5 where R ¼ b(N 2 1)/2c 2 L 2 i p , and P½N  1  q¼ P½q; bðN  1Þ=2cþ 1  q ðN  1Þ b . c denote rounding to the next smaller integer. 4. Let l ¼ lði p þ 1Þ lði p þ 2Þ  lði p þ LÞ   be the vector whose elements are the samples of the frequency response at the transition band. The optimisation problem could be then defined to find l(q) samples which minimise the energy of the stopband attenuation: c ¼ E s ¼ 1 2p ð 2pvs vs Pðe jv Þ     2 dv ¼ pSp T where p ¼ pð0Þ pð1Þ  pðN  1Þ   , and the N  N matrix S is given by ½S i; j ¼ 1  v s p ; i ¼ j  sin½v s ði  jÞ pði  jÞ ; otherwise 8 > < > : Since p T ¼ (1/N )W N 21 P, where W N 21 is the IDFT matrix, c can be expressed as cðl Þ¼ 1 N 2 P T W 1 N   T SW 1 N P ¼ 1 N 2 P T W  N SW 1 N P ð2Þ However, additional constraints should be included in order to obtain a nearly-perfect reconstruction system, with low reconstruction errors, and a high value of minimum stopband attenuation (MSA). For practical reasons, the constrains are included in a multi-objective optimisation problem. Accordingly, the design process consists of achieving the following inequalities: cðl Þ 1 ini ð3Þ max v[½0;p ðjT 0 ðe jv ÞjÞ  min v[½0;p ðjT 0 ðe jv ÞjÞ  d ini pp ð4Þ and MSA  MSA ini ð5Þ where the values of 1 ini , d ini pp (amplitude distortion [3]) and MSA ini (minimum stopband attenuation) are the initially fixed goals of the problem. The overall distortion transfer function T 0 (e jv ) for N-PR M-channel CMFBs can be obtained as T 0 ðe jv Þ¼ e jvðN1Þ M P 2M1 k¼0 Pðe jðvkp=MÞ Þ     2 5. Using the optimised values of l(q) and (1), obtain P opt . 6. Calculate the prototype filter coefficients as p T opt ¼ð1=N ÞW 1 N P opt Finally, the corresponding linear-phase is obtained from the zero- phase prototype filter p opt [n], as p lp [n] ¼ p opt [n 2 b(N 2 1)/2c]. In summary, the design consists of finding the samples l of the frequency response P at the transition band that achieve (3), (4) and (5). Note that the search space is limited to a small number (L) of samples. With this technique, the desired initial values 1 ini , d pp ini , and MSA ini must be previously fixed, and the application itself defines the appropri- ate values for each one of them, and as such, the required prototype filter length. The longer the prototype filter, the better the quality parameters. In this sense, if the prototype filter length is properly selected, the proposed formulation results in a solution better than other previously proposed techniques. Otherwise, the algorithm obtains a solution with approximate values to the initial parameters, and to achieve the desired goals, the filter length has to be changed, or some of them have to be relaxed. To attain a good solution, the initial values for the 1 ini , d pp ini and MSA ini parameters may not be too restrictive, and by means of a trial-and-error method, the final solution is improved. Another alternative to properly determine the initial parameters can be performed very quickly by a two-step procedure: first, determining a start-up solution through a previously reported technique, and secondly, improving that(those) parameter(s) of most interest in our application. At this point it is worth noting that for a given filter length, various combinations of initial parameter values 1 ini , d pp ini and MSA ini can provide different and good solutions that hold the initial criteria. Local optimal solutions can be obtained, and in general, it cannot be demon- strated whether or not an obtained prototype filter is the best possible, and whether the solution is unique. Results: To compare the proposed method, the techniques in [3–5] are considered, because they are also based on the frequency sampling approach and have been compared with the best methods reported in the literature. The design of 128-channel MDFT-based FBs is con- sidered, which is one of the examples included in [5]. We use the above techniques [3–5] to generate 2049-length prototype filters. This filter length allows us to implement the analysis (transmitting for TMUX) and the synthesis (receiving) filters using a fast implementation. ELECTRONICS LETTERS 19th June 2008 Vol. 44 No. 13