IEEE SIGNAL PROCESSING LETTERS, VOL. 17, NO. 6, JUNE 2010 603 Stability Analysis for the Generalized Sidelobe Canceller J. Philip Townsend, Member, IEEE, and Kevin D. Donohue, Senior Member, IEEE Abstract—One approach for reducing noise from non-target locations in array beamformer applications is to use an adap- tive noise cancellation algorithm referred to as the Generalized Sidelobe Canceller (GSC). An analysis is performed to derive a closed-form expression showing the relationship between stability, the number of array channels, adaptive step size , and forgetting factor . The result is verified by applying the GSC to experi- mental data with a range of values for , and array channels to show when the GSC becomes unstable. Index Terms—Adaptive filters, array signal processing, least mean square filters, microphone arrays, noise cancelation. I. INTRODUCTION T HE Generalized Sidelobe Canceller (GSC) is an adaptive noise cancellation system used in array processing sys- tems [1], [2]. The noise reference signal is obtained from the array signals by effectively nulling out the desired signal via a blocking matrix and filtering the array channels with adaptive normalized least mean square (NLMS) filters. While parameter values for the adaptive algorithm have been used in practice that have resulted in stability [3], no general relationship between the system and adaptation parameters has been presented. Hence this relationship is derived here and the stability limits are ver- ified with a series of applications to multichannel speech data recorded in a cocktail party environment. A signal flow diagram of the GSC is presented in Fig. 1 for input data matrix where each matrix column holds a vector of data samples from a particular sensor that has been appropri- ately steered toward the desired focal point with time delays. The upper branch of the system is a delay-sum beamformer (DSB) where are usually set to with the number of sensors. Meanwhile, the lower branch runs the sensor data through a Blocking Matrix (BM) intended to estimate noise ref- erences for the sensor environment. The simplest BM is to take pairwise differences of tracks yielding BM output vectors where the target signal components in each track are assumed to be identical. An integer delay is usually applied to the DSB Manuscript received February 24, 2010; revised April 12, 2010. Date of pub- lication April 26, 2010; date of current version May 07, 2010. The associate editor coordinating the review of this manuscript and approving it for publica- tion was Prof. Yimin Zhang. The authors are with the Center for Visualization and Virtual Environments, University of Kentucky, Lexington, KY 40507 USA (e-mail: jptown0@engr. uky.edu; donohue@engr.uky.edu). This paper has supplementary downloadable material available at http://iee- explore.ieee.org, provided by the authors. This includes all Matlab code used for the Computer Verification section and the recorded voice data used to excite the system as a multichannel WAV file. Digital Object Identifier 10.1109/LSP.2010.2049159 Fig. 1. Generalized Sidelobe Canceller. branch to make up for the signal processing delay encountered by the adaptive filtering in the BM branch. The GSC output is computed from the array channels with the following equation: (1) where is the fixed beamformer output, is the th vector of adaptive filter weights for each blocking matrix output, and is the th blocking matrix output tracks window of length . The filter weights for all output channels are updated using: (2) where is the forgetting factor is the squared Euclidean norm, and is the step size parameter . The parameter determines the magnitude of the filter tap changes every iteration. Large values of result in rapid convergence to- ward a steady-state signal with large misadjustment (variations around the ideal Wiener filter taps), whereas small values re- sult in slow convergence with small misadjustment. The forget- ting factor affects the influence of previously calculated tap weights on the future weights. Choosing is useful when the signal spectra aren’t wide sense stationary (WSS) such as in beamforming on one human voice against several others where the noise properties vary over time. This choice for also pro- vides some robustness for finite precision implementation since it limits the accumulation of quantization errors from previous calculations [4]. 1070-9908/$26.00 © 2010 IEEE