860 IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 5, NO. 4, JULY 2018 DOA Estimation Based on Sparse Representation of the Fractional Lower Order Statistics in Impulsive Noise Sen Li, Rongxi He, Member, IEEE, Bin Lin, Member, IEEE, and Fei Sun Abstract—This paper is mainly to deal with the problem of direction of arrival (DOA) estimations of multiple narrow-band sources impinging on a uniform linear array under impulsive noise environments. By modeling the impulsive noise as α- stable distribution, new methods which combine the sparse signal representation technique and fractional lower order statistics theory are proposed. In the new algorithms, the fractional lower order statistics vectors of the array output signal are sparsely represented on an overcomplete basis and the DOAs can be effectively estimated by searching the sparsest coefficients. To enhance the robustness performance of the proposed algorithms, the improved algorithms are advanced by eliminating the frac- tional lower order statistics of the noise from the fractional lower order statistics vector of the array output through a linear transformation. Simulation results have shown the effectiveness of the proposed methods for a wide range of highly impulsive environments. Index Terms—α-stable distribution, direction of arrival (DOA), fractional lower-order statistics, impulsive noise, sparse represen- tation. I. I NTRODUCTION D IRECTION of arrival (DOA) estimation of multiple emit- ting sources is an important issue in array processing and has various applications in military, radar, sonar, wireless com- munications and source localization [1], [2]. A large number of solutions have been proposed to solve this problem during the past years. Usually, these solutions can be categorized into three groups: time-delay based methods, beamforming methods and signal subspace methods. However, the majority of DOA estimation algorithms are developed under certain as- sumptions: the source signal needs to be statistically stationary and uncorrelated, the number of snapshots is sufficient, and the signal-noise ratio (SNR) is moderately high. Practically, these conditions are barely satisfied, thus these methods achieve the limited estimation accuracy. In order to increase the DOA estimation accuracy, the well-known subspace-based method Manuscript received September 11, 2015; accepted February 18, 2016. This work was supported in part by the National Natural Science Foundation of China (61301228, 61371091) and the Fundamental Research Funds for the Central Universities (3132014212). Recommended by Associate Editor YangQuan Chen. (Corresponding author: Rongxi He.) Citation: S. Li, R. X. He, B. Lin, and F. Sun, “DOA estimation based on sparse representation of the fractional lower order statistics in impulsive noise,” IEEE/CAA J. of Autom. Sinica, vol. 5, no. 4, pp. 860-868, July 2018. All the authors are with the Department of Information Science and Technology, Dalian Maritime University, Dalian 116026, China (e-mail: listen@dlmu.edu.cn; hrx@dlmu.edu.cn; binlin@dlmu.edu.cn; sunfei@dlmu. edu.cn). Digital Object Identifier 10.1109/JAS.2016.7510187 of multiple signal classification (MUSIC) algorithm and esti- mation method of signal parameters via rotational invariance techniques (ESPRIT) have been widely used due to their high estimation accuracies but at the price of high complexity. Recently, the sparse representation technique of signal has been applied in many areas, such as image processing, wireless channel estimation and biomedical signal processing, which also provides a new idea for DOA estimation based on the fact that the number of sources is in general much smaller than that of potential source points when implementing the array processing algorithms. Several DOA estimation methods based on sparse representation have been proposed [3]−[16]. In [3], [4], a whiten sparse covariance-based representation model is first presented for the source parameter estimation by applying the global matched filter (GMF). In [5] the most representative sparse recovery algorithm for DOA estimation (l 1 -SVD) is proposed, which can effectively estimate DOA with single measurement. By using the singular value de- composition (SVD) of received data matrix, it not only can work in multiple measurement cases but also can reduce the computational complexity. Although the l 1 -norm minimization is a convex problem and the global minima can be guaranteed easily, the weakness is their undemocratic penalization for larger coefficients, which results in the degradation of signal recovery performance. To conquer this problem, the iterative reweighted l 1 minimization is designed [6], [7], where the large weights can be used to discourage nonzero entries in the recovered signals. To improve the convergence rate and estimation accuracy of the l 2,1 -norm minimization approach, Wei et al. develop a novel greedy block coordinate descent (GBCD) algorithm by using a greedy strategy for choosing descent directions [8]. In [9], a mixed l 2,0 -norm based joint sparse approximation technique is introduced into DOA es- timation where the l 0 norm constraint is approached by a set of convex functions, and a method called JLZA-DOA is proposed. Algorithms in [5]−[9] address the DOA estimation problem by directly representing the array output in time domain with an overcomplete basis from the array response vector. To make use of the second order statistics of the array output, a sparse iterative covariance-based estimation (SPICE) approach for array signal processing by the minimization of a covariance matrix fitting criterion which can be used in both single and multiple measurement cases is proposed in [10]. Another method called l 1 -SRACV in [11] is also proposed for DOA estimation by using the array covariance