0018-9545 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2015.2504381, IEEE Transactions on Vehicular Technology 1 Performance Analysis of Digital Communication Systems over α-η -μ Fading Channels Osamah S. Badarneh, Member, IEEE, and Mohammed S. Aloqlah Abstract—This paper analyzes and evaluates the performance of digital communication systems that operate over the α-η-μ fading channels. More specifically, we derived novel, unified, and exact closed-form analytical expressions for the cumulative density function (CDF), the moment generating function (MGF), the average channel capacity, and the average symbol error probability (SEP) for several coherent and non-coherent modulation schemes. Note that, the derived expressions are valid for arbitrary values of the fading parameters. The derived expressions are then used to study the implication of the fading parameters on the system performance. In addition, the performance over other well-known fading channels such η-μ and α-μ, and their inclusive special cases can be analyzed using our results. To validate the correctness of our derivations, the numerical results are compared with Monte-Carlo simulation results. Both results are in perfect agreement over a wide range of average signal-to-noise ratio (SNR) and different values of the fading parameters. Index Terms—Average channel capacity, symbol error probability, moment generating function, outage probability. I. I NTRODUCTION I N WIRELESS communication systems, the overall system performance is highly affected by the statistical modeling and characterization of the wireless channel between the transmitter- and the receiver-side. Therefore, modeling of such a wireless channel plays an important role in the design and in the per- formance evaluation of communication systems. In the literature, there are different channel models that accurately describe differ- ent types of phenomena such as multi-path, shadowing, large- and small-scale fading [1]. In the literature, there are many different performance metrics that can be used to analyze communication systems, for example, outage probability, average symbol error probability (SEP), average bit error probability (BEP), amount of fading, and average channel capacity. Generally, obtaining closed-form expressions for the previously mentioned metrics is a challenging problem. However, this challenge becomes very difficult when dealing with complicated generalized fading channel models such as the α-λ-μ 1 , the α-η-μ, the α-κ-μ, the α-λ-μ-η 1 , the η-λ-μ 1 , the λ-μ 1 , the κ-μ and the η-μ fading channels [2–7]. Copyright (c) 2015 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org. Osamah S. Badarneh is with Electrical Engineering Department, University of Tabuk, email: obadarneh@ut.edu.sa, and Mohammed S. Aloqlah is with the Telecommunication Engineering Department, Yarmouk University, email: mohamads@yu.edu.jo . 1 As pointed out by one of the anonymous reviewers, the following clarifications are due: (i) the α-λ-μ channel reported in [2] is exactly the α-η-μ Format II; (ii) the α-λ-μ-η [3] channel is exactly an α-η-μ; (iii) the η-λ-μ channel reported in [4] is exactly an η-μ; (iv) the λ-μ channel reported in [5] is exactly the η-μ channel Format II. All these equivalences come from the fact that power imbalances or correlation between in-phase and quadrature components or even mixed combinations of these lead to the same functional form of the resulting envelope PDF. During the past years, there has been several research stud- ies on evaluating the performance of wireless communication systems over various types of fading channels [8–15]. A closed- form expression for the outage probability over the η-μ fading channels is derived and evaluated in [8]. The authors in [9,10], studied the average channel capacity of single-branch receiver in generalized fading scenarios. The derived expressions are expressed in terms of infinite-series and Meijer’s G-function. In [11], the authors have proposed several approximations to derive approximate expressions for the average SEP and the average channel capacity in terms of infinite/finite-series and Meijer’s G- function. Note that in [9–11], and in order to evaluate the average SEP and the average channel capacity a finite number of terms must be retained and the remaining terms must be truncated. And thus, a truncation error is introduced. In addition, to meet a given accuracy the number of terms in the series is highly depend on the values of the fading parameters. The authors in [12–14] employed the moment generating function (MGF) based approach to derive analytical expressions for the average BEP over the α-μ, η-μ and κ-μ fading channels. In [15], Ermolova and Tirkkonen introduced an approximate technique for reducing the α-η-μ distribution to the well-known generalized gamma distribution. Then based on this approximation, they derived approximate expressions for MGF and the average channel capacity [15]. Note, however, that exact analytical expressions can be obtained if one follows similar analysis presented in this paper. In this paper, we present novel exact closed-form expressions for the outage probability, the MGF, the average SEP/average BEP for several coherent and non-coherent modulation schemes, and the average channel capacity over the α-η-μ fading chan- nels. Our derived expressions are valid for arbitrary values of the fading parameters, namely, α, η, and μ. Furthermore, other well- known fading distributions, and their inclusive ones, are derived from our results as special cases such as, the η-μ, and the α-μ distributions. Our derived expressions are represented in terms of the well-known bivariate Fox’s H-function (BFHF) (also known as Fox’s H-function of two variables). To summarize, the contribution of this paper is as follows: 1) Novel exact closed-form statistical expressions for the outage probability, MGF, average SEP, and average channel capacity are provided. 2) The analysis in this paper is unified in the sense that it can be applied for other similar fading channels whose envelope follows the form of x β exp(ξx α )I v (ζx α ). For example, the η-μ and the α-η-μ distributions [3–5, 7]. 3) We provide useful results for Laplace transform, which are not reported in the literature. These results can be useful in many practical applications, for example, evaluating the average SEP, the average channel capacity, and the MGF. The remainder of this paper is organized as follows: In Section