Available online at www.sciencedirect.com Mathematics and Computers in Simulation 80 (2009) 270–293 Detection-recognition algorithm based on the Gabor transform for unknown signals embedded in unknown noise Ewa Swiercz a,∗ , Andrzej Pieniezny b,1 a Bialystok Technical University, Faculty of Electrical Engineering, Wiejska 45D, 15-351 Bialystok, Poland b Military University of Technology, Faculty of Electronics, Kaliskiego 2, 00-908 Warsaw, Poland Received 21 April 2006; received in revised form 25 October 2008; accepted 14 June 2009 Available online 24 June 2009 Abstract In this paper we present the CFAR (Constant False Alarm Rate) two-step detection-recognition algorithm for unknown, non- stationary signals embedded in unknown noise, based on the discrete Gabor transform. In the detection step, the decision about the absence or the presence of a signal of interest in a background of noise should be taken. The term ‘recognition’ means recovering the signal waveform from a noisy signal after the detection step. The recognition can be reformulated as the non-stationary, time-varying filtering problem in a time–frequency domain. In this paper the Gabor time–frequency domain is taken into account and the Gabor transform is used both in the detection and the in the filtering step. The discrete Gabor transform (DGT) is under intensive study of mathematicians, what results in a number of new, efficient computational algorithms for long time series. The Gabor frame approach is used for computation analysis and synthesis windows. Data-driven approach to develop the detection-recognition algorithm is based on the assumption, that disturbing noise signal after the Gabor transform, can be successfully approximated by the Weibull distribution regardless noise distribution before the transformation. It is shown by intensive simulations, that a two-parameter model like the Weibull distribution is really appropriate. Scale and shape parameters of the Weibull distribution are easily estimated and the CFAR threshold used in detection, based on estimated parameters, can be computed. The case of a low SNR ratio, with additional assumption about a signal, is also considered. It is shown that the iterative form of the time-varying filtering, significantly improves the quality of the whole detection-recognition CFAR algorithm. This approach is successfully investigated on a real-life radar signal. © 2009 IMACS. Published by Elsevier B.V. All rights reserved. Keywords: Gabor transform; CFAR detection; Time-variant filtering 1. Introduction The classical optimal detection of signals embedded in noise consists of choosing one hypothesis from a set of hypotheses, based on the knowledge of certain underlying statistics. In most practical applications these statistics are not available, since the phenomena related to these hypotheses are complex and poorly understood. In such situations approximations of optimal detectors have to be developed. The alternative way of developing the detection strategy is deriving a detector based directly on received data. When a signal is non-stationary, the representation in terms of ∗ Corresponding author. Tel.: +48 85 7469 361; fax: +48 85 7469 360. E-mail addresses: ewasw@pb.edu.pl (E. Swiercz), apieniezny@wel.wat.edu.pl (A. Pieniezny). 1 Tel.: +48 22 6837 334; fax: +48 22 6837 461. 0378-4754/$36.00 © 2009 IMACS. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.matcom.2009.06.028