IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 46, NO. 6, JUNE 1999 649 Envelope-Constrained IIR Filter Design via Optimization Methods Zhuquan Zang, Member, IEEE, Antonio Cantoni, Fellow, IEEE, and Kok Lay Teo, Senior Member, IEEE Abstract— Previously, the envelope-constrained (EC) filtering problem was formulated as designing an infinite impulse response (IIR) filter such that the filter’s norm is minimized, subject to the constraint that its response to a specified input pulse lies within a prescribed envelope. In this paper we recast this filter design problem as a frequency-domain optimization problem with time-domain constraints. Motivations for solving this problem are given. Then, the recently developed opti- mization techniques are used for the design of the required IIR filter. For illustration, we apply the approach to two numerical examples which deal with the design of equalization filters for digital transmission channels. Index Terms—Digital transmission channel, envelope-con- strained filter, equalization filter, optimization, IIR filter, minimax problem, model matching theory, state space. I. INTRODUCTION I N signal processing many filter design problems often can be cast as a constrained optimization problem where the constraints are defined by the specifications of the filter. These specifications can arise either from practical considerations or from the standards set by certain regulatory bodies (see, e.g., [3]). In this contribution, we are concerned with the envelope-constrained (EC) filtering problem. Our objective is to design a linear time invariant (LTI) filter with impulse response to process a given input signal which is corrupted by additive random noise [see Fig. 1(a)]. The noiseless output is required to fit into a prescribed pulse shape envelope defined by the lower and upper boundaries and [see Fig. 1(b)]. Previously [4], the optimal EC filter was defined as the filter which minimizes the output noise power while satisfying the pulse shape constraints. Assuming that the random noise is white with constant power spectrum density, it can be verified that the output noise power is proportional to the squared norm of the filter to be designed. Hence, the optimal EC filtering problem can be posed as subject to (1) where is the squared norm of the filter’s impulse response. Manuscript received September 16, 1996; revised May 18, 1998. This work was supported in part by a research grant from the Australian Research Council. This paper was recommended by Associate Editor V. Tavsanoglu. The authors are with the Australian Telecommunications Research Insti- tute, Curtin University of Technology, Perth WA 6001, Australia (e-mail: zang@atri.curtin.edu.au). Publisher Item Identifier S 1057-7122(99)04742-X. (a) (b) Fig. 1. EC filtering problem: (a) block diagram and (b) pulse shape envelope. In standards, the performance of a digital link is often specified in terms of a mask applied to the received signal [2], [3], [6]. The EC filter design problem is directly applicable and the input signal would correspond to the test signal specified in the standard. Other areas of application include robust antenna and filter design [1] and pulse compression in radar and sonar [7]. Originally [4], the optimal EC filter design problem was posed in the continuous-time domain as an space opti- mization problem. The discretized version of this problem has been solved using finite impulse response (FIR) filter structure and various optimization techniques, see, e.g., [4] and [13]. Although FIR filters are attractive due to their simplicity, they generally require a large number of taps and the number of taps needed in general is highly sensitive to the sampling rate. In this contribution, using the recently developed con- strained robust control design techniques [10], [12], we design an infinite impulse response (IIR) filter such that its norm, defined as (see, e.g., [5]), is minimized subject to the same time-domain constraints as specified in (1). The use of the norm arises naturally when the power spectrum of the exogenous input noise is bounded, but otherwise unknown, while the use of the norm in the EC filtering problem is relevant when the power spectrum of the exogenous input noise is known. We shall demonstrate that, combined with suitable model reduction techniques, the use of the norm and an IIR filter structure offer a more robust low-order alternative to FIR filters. 1057–7122/99$10.00  1999 IEEE Authorized licensed use limited to: CURTIN UNIVERSITY OF TECHNOLOGY. Downloaded on June 10, 2009 at 10:02 from IEEE Xplore. Restrictions apply.