IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 48, NO. 5, SEPTEMBER 1999 1381 Performance Analysis of Digital Cellular Radio Systems in Nakagami Fading and Correlated Shadowing Environment Mohammed Abdel-Hafez, Student Member, IEEE, and Mehmer ¸ Safak, Senior Member, IEEE Abstract— The bit error rate (BER) performance of digital cellular radio systems was investigated in a Nakagami fading, correlated lognormal shadowing and additive white Gaussian noise (AWGN) environment for noncoherent differential phase shift keying (DPSK) modulation. Two models were used to deter- mine the BER; the first one is based on the cumulative power levels of cochannel interferers while the second one is based on instantaneous cochannel interference power. The relative ad- vantages of the two models were presented for various design parameters. The effects of bit energy to noise ratio, frequency reuse distance, cluster size, correlation coefficient, shadow spread, and fading parameter were studied. The BER was observed to be lower in a correlated shadowing environment compared with the uncorrelated case. The near–far effect was studied by assuming that both the desired and interfering mobiles are randomly located in their corresponding cells. Index Terms— Cellular radio system, cochannel interference, correlated shadowing, multipath fading. I. INTRODUCTION T HE DEMAND for an increased number of subscribers in cellular radio systems is rapidly growing. The design ob- jective for these systems is therefore to improve the efficiency of the allocated spectrum usage (increase the number of users per unit bandwidth), subject to constraints on complexity and quality of service. Digital systems can potentially offer better channel utilization and better quality services than the analog systems. The wave propagation in cellular radio systems is modeled by the path loss, cochannel interference, multipath fading, and shadowing. The path loss between the mobile unit and the base antenna is usually modeled by a dual-slope curve [1], where the received power is attenuated by at distances shorter than a turning point, and beyond it. The turning point is determined by transmitting and receiving antenna heights, and the operating frequency. Different approaches were used to consider the effect of cumulative cochannel interference [1]–[4]. The total inter- ference power is usually determined by the sums of the powers of the additive white Gaussian noise (AWGN) and of the instantaneous cochannel interferers. In [1] and [3], the Manuscript received July 30, 1996; revised August 26, 1998. M. Abdel-Hafez is with Center for Wireless Communications, University of Oulu, 90401 Oulu, Finland. M. ¸ Safak is with the Department of Electrical and Electronic Engineering, Hacettepe University, 06532 Beytepe, Ankara, Turkey. Publisher Item Identifier S 0018-9545(99)05749-7. signal-to-interference ratio (SIR) is determined by the sums of the powers of the AWGN and of the local-mean levels of cochannel interferers (nonfaded interferers). The second model in based on the sums of the instantaneous received powers of the individual cochannel interferers instead of their mean values (faded interferers) [2]. This paper provides a comparative study on the relative merits of these two models. The multipath fading in cellular radio channel is described by Nakagami -distribution [5], which can approximate dif- ferent fading environments including those characterized by Rayleigh and Rician fading. Since it is mathematically more convenient than Rician distribution, we have chosen to use the Nakagami distribution for our calculations. The major disadvantage of Nakagami distribution compared with Rician distribution is in differences in tail behavior which results in a major effect on our results. The fading parameter of the Nakagami -distribution can describe the presence or absence of the line of sight (LOS) between the user terminal and the base station. In macrocel- lular systems, the cell diameter is usually 2–20 km, and the antenna radiating power is in the order of 0.6–10 W from high towers. In such channels, the LOS is usually blocked, which means that the fading parameters of the desired signal and of the interferers are In microcellular systems, the antenna height is few meters with radiating power less than 20 mW and cell diameter of 0.4–2 km. In such systems, the desired signal usually has a LOS component (the fading parameter while the cochannel interferers, with LOS blocked, have In indoor channels, the coverage will be provided by the picocells with a cell diameter of 15–200 m, with antenna radiating powers of few microwatts. In these systems, the desired signal and the interferers might have LOS component, but the fading parameter of the desired signal is expected to be larger than those of the interferers. The analysis presented here is valid for all types of cellular systems. The received local-mean signal power level is usually influenced by the natural and man-made obstructions, such as hills, trees, or vegetation. Such phenomenon is called shadowing (slow changing in the local-mean power level). When the same obstacles shadow various cochannels inter- ferers, the local-mean power levels of the shadowed signals become correlated with each other [6], [7]. The correlation between the local-mean power levels of the received signals depends on their angles of arrival and the propagation path 0018–9545/99$10.00  1999 IEEE