Published in IET Signal Processing Received on 15th November 2009 Revised on 1st April 2010 doi: 10.1049/iet-spr.2009.0272 ISSN 1751-9675 Spectrally constrained channel shortening using least-squares optimisation Md. Kamrul Hasan 1,2 M.A. Haque 1 T. Islam 1 1 Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh 2 Department of Biomedical Engineering, Kyung Hee University, South Korea E-mail: khasan@eee.buet.ac.bd Abstract: Channel shortening by least-squares (LS) optimisation is an attractive technique for its simplicity and computational efficiency. However, the method does not have suitable control over the frequency response of the shortened channel. As a result, the deep spectral nulls may inhibit some subcarriers to carry data bits and thereby reduce bit rate in multicarrier communication systems. Channel shortening is also proposed as a potential dereverberation technique in some recent research results. Again, shortening by LS optimisation leads to severe spectral distortion in the dereverberated speech signal. In this paper, we propose a spectrally constrained iterative LS minimisation algorithm that enforces spectral flatness in the shortening filter and thereby removes nulls without sacrificing the shortening performance. We also propose an optimal step-size for the iterative LS technique, which yields the fastest convergence rate for the gradient descent algorithm. The effectiveness of the proposed algorithm is tested for asymmetric digital subscriber line channels and speech dereverberation problems. The simulation results show that it outperforms the conventional techniques, resulting more subcarriers to carry bits when applied to communication channels and better quality of the speech signal when acoustic channels are considered. 1 Introduction Channel shortening concept is traditionally linked with the equalisation of communication channels. It is a well-known technique to combat intersymbol/intercarrier interference caused by the inadequate cyclic prefix length in multicarrier transmission systems [1], such as for discrete multitone (DMT)-based asymmetric digital subscriber line (ADSL) system or orthogonal frequency division multiplexing for wireless communications. Recently, channel shortening has been proposed as an emerging technique for speech dereverberation [2] in order to facilitate the advent of modern hands-free speech communication systems. From psychoacoustic perspective, the acoustic channels does not require complete equalisation and, therefore, a shortened channel that requires less computation with acceptable performance can serve the purpose. The shortening of a long impulse response is usually performed using a finite impulse response (FIR) filter known as time-domain equaliser (TEQ ). The TEQ design problem for communication channel has been extensively studied in the literature, among which least- squares (LS) optimisation-based methods are very common [3–7]. Minimum mean-squared error (MMSE) TEQ design method proposed in [4] minimises the time- domain error between the TEQ output and a desired output. In [5], Melsa et al. proposed the maximum shortening SNR (MSSNR) method, which attempts to maximise the ratio of the energy inside a target window to the energy outside the window. This approach was generalised in the min-ISI method reported in [6], which allows the residual inter-symbol interference (ISI) to be shaped in the frequency domain. However, the MMSE, MSSNR and min-ISI methods exhibit deep spectral nulls in the frequency domain, and will likely clobber a good subchannel [7]. A low complexity eigenfilter method is presented in [8] considering the minimum delay spread (MDS) of the effective channel as well as the effect of noise. In [9], an improved version of this method is 698 IET Signal Process., 2010, Vol. 4, Iss. 6, pp. 698–707 & The Institution of Engineering and Technology 2010 doi: 10.1049/iet-spr.2009.0272 www.ietdl.org