1560 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 4, APRIL 2007 Adaptive Detection With Bounded Steering Vectors Mismatch Angle Olivier Besson, Senior Member, IEEE Abstract—We address the problem of detecting a signal of interest (SOI), using multiple observations in the primary data, in a background of noise with unknown covariance matrix. We consider a situation where the signal’s signature is not known perfectly, but its angle with a nominal and known signature is bounded. Furthermore, we consider a possible scaling inhomogeneity between the primary and the secondary noise covariance matrix. First, assuming that the noise covariance matrix is known, we derive the generalized-likelihood ratio test (GLRT), which involves solving a semidefinite programming problem. Next, we substitute the unknown noise covariance matrix for its estimate obtained from secondary data, to yield the final detector. The latter is compared with a detector that assumes a known signal’s signature. Index Terms—Array processing, detection, generalized-likelihood ratio test (GLRT), steering vector mismatch. I. INTRODUCTION AND PROBLEM STATEMENT The problem of detecting the presence of a signal of interest (SOI) against a background of colored noise is fundamental, especially in radar applications. This problem has been studied extensively in the lit- erature (see, e.g., [1] for a list of references). Usually, the presence of a target is sought in a single vector under test, assuming that training sam- ples, which contain noise only, are available (see, e.g., [2] and [3]). Fur- thermore, the space or space–time signature of the target is assumed to be known. Herein, we consider a slightly different framework, namely we assume that the primary data contains multiple observations and that the SOI signature is not known perfectly. The first assumption arises, for instance, when a high-resolution radar attempts to detect a range-spread target [4]. In such a case, the primary data consists of the array outputs in the range cells in which the target is likely to be present. Uncertainties about the SOI signature can be attributed to many pos- sible causes, including uncalibrated arrays, pointing errors, multipath propagation, and wavefront distortions [5], [6]. Detection with multiple observations in the primary data and partly known signals of interest has been considered, among others in [4], [5], and [7]–[9]. Reference [5] considers detecting uncertain rank-one waveforms when both the space and time signatures of the SOI are assumed to belong to known linear subspaces. Bose and Steinhardt de- rive the maximal invariant along with the generalized-likelihood ratio test (GLRT) for this general framework. An extension to partially ho- mogeneous environments is considered in [9]. Reference [4] considers the detection of a range-spread target using a high-resolution radar, as- suming that the steering vector is known. The authors derive and an- alyze a two-step GLRT for both the homogeneous and the partially homogeneous case. In [7] the SOI signature is considered as unknown and arbitrary. The theory of invariance is invoked to obtain a most pow- erful invariant test and a suboptimal constant false alarm rate (CFAR) detector. Reference [8] proposes a CFAR detector based on a two-step GLRT when the signals of interest are Gaussian and belong to a known subspace. Manuscript received January 11, 2006; revised June 28, 2006. The associate editor coordinating the review of this manuscript and approving it for publica- tion was Dr. Daniel Fuhrman. O. Besson is with the Department of Avionics and Systems, ENSICA, 31056 Toulouse, France (e-mail: besson@ensica.fr). Digital Object Identifier 10.1109/TSP.2006.890820 For most of the above-mentioned studies, the fact that the SOI lies in a subspace facilitates the derivation of the detectors, as one usually ends up with closed-form detectors. However, the choice of this subspace is delicate since one must ensure that the SOI really belongs to it. Other- wise, there is some loss of performance. In this paper, we investigate a different approach. We assume that we have knowledge of the nominal value of the SOI signature and that the actual signature is “close” to its nominal value. More precisely, we assume that the angle between these two vectors is bounded. This approach was already advocated by the author in [10]. However, the latter reference only considers the case of a single vector under test, viz. : the present paper is an extension of [10] to the case where the primary data contains multiple snapshots. As will be illustrated below, considering involves considerable complications, notably in the derivation of the maximum-likelihood es- timate (MLE) of the SOI’s signature. When , the technique of Lagrange multipliers (with a single Lagrange multiplier) is invoked in [10] to obtain the maximum-likelihood (ML) estimator. It is shown in [10] that the Lagrange multiplier is the unique solution to a secular equation. When , finding the MLEs turns out to be more com- plicated as it involves maximizing a quadratic form with an equality constraint and a nonconvex quadratic inequality constraint. In this case, the problem can be written as a semidefinite program, as will be shown in the next section. We now define formally our detection problem. It consists of de- ciding between the following two hypotheses: (1) We denote by the primary data array and , the secondary data array. stands either for a space snapshot—in which case denotes a time index—or a space–time snapshot, with being the number of array antennas times the number of pulses. In the latter case, corresponds to a range index. In (1), and stand for the noise in the primary and sec- ondary data, respectively. We assume that they are proper, zero-mean, independent, and Gaussian-distributed random vectors with covariance matrices (2a) (2b) Hence, we consider a partially homogeneous environment in which the covariance matrices of the primary and the secondary data have the same structure, but possibly different power. Both and are un- knowns. Under , the SOI is a rank-one matrix , with . is either the spatial or the space-time sig- nature of interest, referred to as the steering vector, while denotes the amplitude of the SOI in the time or range domain. We assume that is arbitrary and unknown. Although is unknown, we assume that it is close to a nominal (and therefore known) steering vector . To account for this uncertainty, we assume that is mostly aligned with , and that the fraction of its energy outside is bounded. More precisely, it is assumed that (3) where is a scalar that sets how much of the energy is allowed to be outside . The constraint (3) means that the square 1053-587X/$25.00 © 2007 IEEE