Modulation Classification Based on Gaussian Mixture Models under Multipath Fading Channel Jason Gejie Liu * , Xianbin Wang * , Jay Nadeau * , and Hai Lin † * Dept. of Electrical and Computer Engineering, The University of Western Ontario, Canada † Dept. of Electrical and Information Systems, Osaka Prefecture University, Japan Email: {gliu63, xianbin.wang, jnadeau6}@uwo.ca, lin@eis.osakafu-u.ac.jp Abstract—This paper considers the classification of digital modulation schemes in the presence of multipath fading channels and additive noise. A novel modulation recognition approach is proposed based on Gaussian Mixture Models (GMM). Our basic procedure involves parameter estimation using GMM to set up an offline database and then to classify the received signal into different modulation schemes based on the database by using Kullback-Leibler (K-L) Divergence. In order to mitigate the negative impact from multipath fading channels, an iterative Maximum A Posteriori (MAP)-based channel estimation is used in conjunction with the Expectation-Maximization (EM) algo- rithm. Furthermore, Gaussian approximation is carried out to decrease the computational complexity. Monte Carlo simulations are conducted to evaluate the performance of individual mod- ulation scheme classification. Numerical results show that the proposed approach is capable of recognizing various modulated signals with improved performance under AWGN and multipath fading channels. I. I NTRODUCTION Modulation classification is a technique used to automat- ically identify the modulation scheme of a communication signal by observing and analyzing its statistical properties. The technique was originally used for signal interception in military communications but has received renewed interest recently in other areas including adaptive modulation, software defined radio and cognitive networks.To identify the modula- tion of an incoming signal, decision-theoretic methods [1]- [3] and pattern recognition [4]- [6] are two primary solutions, both of which involve the following steps: preprocessing of the signal and proper selection of the classification algorithm. Decision-theoretic approaches are based on the likelihood function or approximation theory [2]- [3]. Although the decision-theoretic classifiers with maximum likelihood (ML) are optimal, the corresponding closed-form solutions are either unavailable or computationally intensive when performing a numerical search. In addition, these approaches are not robust with respect to the model mismatch in the presence of a fading channel as well as phase or frequency offsets. Pattern recognition methods [4]- [6] may not be optimal but are generally simple to implement for near optimal per- formance if designed appropriately. Under this category, the modulation classification consists of two subsystems: fea- ture extraction and pattern recognition. Commonly adopted techniques are higher-order statistics (HOS), including cyclic cumulants [4]- [5] and statistical moments [6]. Although these methods can be robust when dealing with model mismatch, the classification performance under low signal-to-noise ratio (SNR) scenarios is not satisfactory and the computational complexity can be high. To alleviate the computational load while maintain classification performance, we propose a recog- nition method in this paper based on Gaussian Mixture Models (GMM) to identify the modulation schemes. Moreover, a channel impact mitigation approach is also introduced in the presence of multipath fading channels . GMM has been used successfully in areas such as sta- tistical analysis and speech processing, by representing the distribution of the signal of interest with a weighted sum of several multivariate Gaussian functions. The parameters in the model are the weights, mean values and covariances, which can be estimated using the Expectation-Maximization (EM) algorithm. Since the estimation is based on statistical charac- teristics, when it comes to wireless propagation, the approach is not sensitive to the transmission schemes, such as dissimilar carrier frequencies, sampling frequencies or symbol rates, thus, making it more robust for implementation. For modulation classification purpose, a GMM-based offline database needs to be established, which contains the parameters for different modulation schemes as the reference to determine the GMM parameters of the received signal. Previous related work has been investigated in [7] without considering multipath fading channels. In this paper, an iterative Maximum A Posteriori (MAP) channel mitigation [9] technique is introduced to mitigate the multipath fading as well as maintain system performance. Kullback-Leibler (K-L) Divergence is employed to measure the distance between the received signal and the modulation schemes in the database. To further ease the computational complexity, Gaussian approximation [8] is carried out to cope with multivariate Gaussian components. Performance analysis is presented using Monte Carlo simula- tion to validate the effectiveness of classification accuracy. The rest of the paper is organized as follows. The GMM and K-L Divergence are introduced in details in Section II. Section III presents the proposed GMM-based modulation classification using the instantaneous amplitude and phase of the received signals. Iterative MAP channel mitigation is proposed in Section IV to deal with multipath fading chan- nel interference. In Section V, Monte Carlo simulations are performed to evaluate the performance and provide the com- parison with HOS-based modulation classification. Finally, the paper is concluded in Section VI. Globecom 2012 - Signal Processing for Communications Symposium 3994