Design of frequency response masking FIR filter in the Canonic Signed Digit space using modified Artificial Bee Colony algorithm Manju Manuel n , Elizabeth Elias Department of Electronics and Communication Engineering, National Institute of Technology, Calicut, Kerala 673601, India article info Article history: Received 17 September 2011 Received in revised form 2 January 2012 Accepted 14 February 2012 Available online 6 March 2012 Keywords: Artificial Bee Colony algorithm Differential Evolution algorithm Frequency response masking filter Canonic Signed Digit Discrete optimization abstract Frequency response masking (FRM) technique along with the Canonic Signed Digit (CSD) representa- tion is a good alternative for the design of a computationally efficient, sharp transition width, high speed finite impulse response (FIR) filter. This paper proposes two novel approaches for the joint optimization of an FRM FIR digital filter in the CSD space. The first approach uses the recently emerged Artificial Bee Colony (ABC) algorithm and the second approach uses the Differential Evolution (DE) algorithm. In this paper, both the algorithms are modified in such a way that, they are suitable for the solution of the optimization problem posed, in which the search space consists of integers and the objective function is nonlinear. The optimization variables are encoded such that they permit the reduction in computational cost. The salient feature of the above approaches is the reduced computational complexity while obtaining good performance. Simulation results show that the ABC based design technique performs better than that using DE, which in turn outperforms the one using integer coded genetic algorithm (GA). The proposed optimization approaches can be extended to the solution of integer programming problems in other engineering disciplines also. & 2012 Elsevier Ltd. All rights reserved. 1. Introduction If the coefficients of a digital filter are quantized to the Signed Power of Two (SPT) space, then it is possible to replace the multipliers by shift and add operations (Hartley, 1996). Removing the multipliers is equivalent to reducing the circuit complexity and minimizing the power dissipation and chip area. In the SPT space, the CSD is a minimal representation as it represents a given decimal number using minimum number of nonzero SPT terms. In CSD representation, subtractions are also used to carry out multiplications for efficient hardware usage. In this scenario, it is quite advantageous to extend the CSD representation to the digital filter designed using frequency response masking (Yu and Lim, 2002a). FRM (Lim, 1986) brings forth tremendous savings in the number of multipliers while designing sharp filters with arbitrary bandwidth. The design of the FRM filter having infinite precision coefficients has been well investigated. It involves the design of an interpolated band edge shaping filter and a pair of masking filters. To this end, several publications are available in which the FRM filter coefficients are obtained either by the separate optimization of the various sub-filters or joint optimization of all of them. Linear programming is used for the design of the sub- filters which minimized the weighted error in the pass band and stop band of the overall filter by Lim (1986). Remez algorithm is used by Saramaki and Lim (2003) for the design of FRM FIR filter. The joint optimization of the various sub-filters is done in the following papers. Weighted Least square method is employed by Yu and Lim (2002b) and Lee et al. (2004). Semi-definite program- ming (Lu and Hinamoto, 2003) and second order cone program- ming (Lu and Hinamoto, 2008) techniques are used for the design of the continuous coefficient FRM filter. Recently, Neural Network has been employed (Wang and He, 2008) for the design of the FRM filter, in which the coefficients of the sub-filters are opti- mized simultaneously. The implementation complexity can be significantly brought down when the FRM filter is represented using CSD. The design of an FRM filter in the discrete space is a complicated process and it calls for the use of efficient nonlinear optimization techniques. The classical gradient based optimization techniques cannot be directly applied to this problem, because here, the search space consists of integers. In this context, meta-heuristic algorithm is a good optimization tool since the proper tuning of the parameters with respect to a particular design problem can bring forth global solution. Genetic algorithms have been established as a good alternative for the optimization of multimodal, multidimensional problem. Contents lists available at SciVerse ScienceDirect journal homepage: www.elsevier.com/locate/engappai Engineering Applications of Artificial Intelligence 0952-1976/$ - see front matter & 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.engappai.2012.02.010 n Corresponding author. Tel.: þ91 4812401131. E-mail addresses: manju.manuel@gmail.com (M. Manuel), elizabeth@nitc.ac.in (E. Elias). Engineering Applications of Artificial Intelligence 26 (2013) 660–668