Alternative algorithm for maximum likelihood DOA estimation and detection A. Swindlehurst lndexing terms: Narrowband direction of arrival estimation, Algorithm, Maximum likelihood methods, Computational complexity, Cost junction Abstract: The majority of algorithms developed for the narrowband direction of arrival (DOA) estimation problem rely on an eigenvalue decom- position (EVD) to determine both the number of signals and their respective DOAs. An alternative algorithm is presented that solves both the DOA detection and estimation problems without resorting to an EVD. The algorithm is shown to have asymptotically equivalent performance to that of the (unconditional) maximum likelihood method, and hence it yields asymptotically minimum variance DOA estimates. The computa- tional complexity required to update the DOA estimates in response to additional data from the array is investigated, and the algorithm is shown to be somewhat simpler than other methods with comparable performance. In addition, the asymp- totic distribution of the algorithm’s cost function is derived, and is shown to be composed of the sum of two differently scaled chi-squared random variables. A hypothesis test for determining the number of signals based on this result is then pre- sented. 1 Introduction Maximum likelihood methods are a standard approach to solving parameter estimation problems such as those encountered in narrowband direction of arrival (DOA) estimation. In the maximum likelihood (ML) approach, the probability distribution of the observations is expressed as a function of the parameters to be estimated, and the so-called (log)-likelihood is maximised with respect to these parameters. Under certain regularity con- ditions, estimates obtained from the ML approach are both asymptotically unbiased and eficient, meaning that they achieve the Cramer-Rao lower bound (CRB) on estimate variance. ML solutions to the DOA estimation problem have been proposed by a number of authors. Two types of solutions have been obtained: one for the case of deter- ministic signals [I-41 (referred to as conditional ML), and one for a stochastic signals model [2, 4-61 (referred to as unconditional ML). Despite the desirable properties of these ML estimators, they have not enjoyed much pract- ical application since they typically require nonlinear, 0 IEE, 1994 Paper 1366F (ES), first received 8th July 1993 and in revised form 1 lth April 1994 The author is with the Department of Electrical and Computer Engin- eering, Brigham Young University, Provo, UT 84602, USA 1EE Proc.-Radar, Sonar Nauig., Vol. 141, No. 6, December 1994 multidimensional optimisation procedures. In the case of deterministic ML, a further drawback is that its estimates do not asymptotically achieve the Cramer-Rao lower bound (CRB) on estimate variance [7]. This is due to the fact that the number of free parameters to be estimated grows with the amount of data collected. The difficulties associated with ML techniques have led to a proliferation of DOA estimation methods, especi- ally those based on the separation of the data into ‘signal’ and ‘noise’ subspaces. These methods rely on an eigen- value decomposition (EVD) of the array covariance matrix, and either a search of a one-dimensional spec- trum [SI or the calculation of the roots of a certain poly- nomial [9]. While these methods achieve ML or near-ML performance in many cases, difficulties arise when the signals have nearly coincident DOAs or are highly correlated. These difficulties are a direct result of using a one-dimensional optimisation procedure to solve what is inherently a multidimensional problem. Recently, two new subspace (EVD) based multidimen- sional algorithms have been proposed whose asymptotic second-order performance is equivalent to that of ML; i.e. they yield DOA estimates that asymptotically achieve the CRB for arbitrary second-order ergodic signals and Gaussian noise. These algorithms are the weighted sub- spacefitting (WSF) method of Viberg and Ottersten [lo, 111, and the method of direction of arrival estimation (MODE) algorithm of Stoica and Shaman [l2, 131 (sometimes referred to as noise subspace fitting (NSF)). Both algorithms rely on a multidimensional search of roughly the same order of complexity as ML to estimate the DOAs. Since in addition both require an EVD, one may legitimately wonder what is gained by implementing them in lieu of ML. A number of empirical studis [ll, 141 have demonstrated one advantage, indicating that WSF has better convergence properties than both condi- tional and unconditional ML. The primary drawback of using algorithms based on the EVD occurs when operating in a tracking or updat- ing mode where, given a small number of additional snapshots from the array, one attempts to recompute the DOA estimates using the most recent estimates and pre- viously received data. For example, updating the WSF and MODE estimates requires that the principal eigen- space be completely recomputed. While algorithms have been proposed for performing the eigenspace update efi- ciently [15-191, such techniques are still quite complex and often suffer from a linear buildup of round-off error. This work was supported by the IR&D Program at ESL, Inc., and by the National Science Founda- tion under grant MIP-9110112. 293 Authorized licensed use limited to: IEEE Editors in Chief. Downloaded on August 17, 2009 at 19:48 from IEEE Xplore. Restrictions apply.