Interference Cancellation for EDGE via Two-User Joint Demodulation Abdulrauf Hafeez * , Dennis Hui * and H¨ useyin Arslan † * Ericsson Inc., 8001 Development Drive, P. O. Box 13969, RTP, NC 27709, USA † University of South Florida, Electrical Eng. Dept., Tampa, FL 33620, USA Email:{abdulrauf.hafeez, dennis.hui}@ericsson.com, arslan@eng.usf.edu Abstract— EDGE has been introduced as an evolution of GSM for providing voice and high-speed data services. To cope with the ever-increasing demands on higher spectral efficiency, a tight frequency reuse (1/3 or even 1/1) may be adopted for GSM/EDGE in the foreseeable future, motivating the need to investigate advanced interference cancellation techniques. In this paper, we consider joint demodulation for cancelling a dominant cochannel or adjacent-channel interferer. A complete receiver structure is described assuming burst-synchronous users. Simulation results show that large performance gains are obtained with moderate receiver complexity in a dominant interferer–limited environ- ment. Index Terms— GSM/EDGE, Joint Demodulation, Interference Cancellation, Reduced Complexity Equalizer, Cochannel Inter- ference, Adjacent Channel Interference I. I NTRODUCTION Enhanced Data Rates for Global Evolution (EDGE) [1] is an evolution of Global System for Mobile Communications (GSM) for providing voice and high speed multimedia ser- vices. EDGE provides up to three times the data rate of GSM by employing 8-PSK modulation in addition to GMSK modu- lation. For high spectral efficiency, EDGE may be introduced in a tight frequency re-use pattern, such as 1/3 or 1/1, resulting in an interference-limited system. This motivates the need for interference cancellation techniques, such as interference rejection combining [2] and multiuser demodulation. The latter technique does not require multiple receive antennas which is desirable for handsets due to cost and size constraints. For GSM, multiuser demodulation has been considered for cochannel interference (CCI) cancellation in [3] and adjacent channel interference (ACI) cancellation in [4]. In this paper, we consider joint demodulation of the desired user and one dominant interferer (cochannel or adjacent channel) for the GSM/EDGE system. We show that a moderate complexity joint demodulator provides large gains over a single-user demodulator in CCI and ACI limited environments. A major challenge in designing a joint demodulator for EDGE is the higher constellation density of 8-PSK mod- ulation as compared to GMSK modulation, which can be approximated as partial-response BPSK. Since the partial- response pulse-shape of GMSK is also used for 8-PSK trans- mission, the channel impulse response spans at least four symbol periods, making joint demodulation using maximum- likelihood sequence estimation (MLSE) prohibitive. As a reduced-complexity alternative, we consider decision feedback sequence estimation (DFSE) [5] which is applied jointly for two users. It is well known that DFSE is effective only when the channel has a minimum-phase (or close to minimum- phase) characteristic. An all-pass pre-filter that converts both users’ channels into minimum-phase does not exist, except in pathological cases. Therefore, we consider a finite impulse response (FIR) pre-filter similar to the impulse response shortening pre-filter of [6] to compact the signal energy in two leading channel taps for both users. This allows joint DFSE demodulation with 64 states when both users employ 8-PSK modulation, 16 states when one user employs GMSK modulation and 4 states when both users employ GMSK modulation. In this paper, we assume that the users are burst- synchronous and that their modulation types, training se- quences and carrier frequency offsets are known at the re- ceiver. We jointly estimate the channel responses for the desired user and the dominant interferer for each burst by using a joint least-squares (LS) algorithm over the training period. Analytical results show that certain training sequence pairs result in large channel estimation error which affects cochannel performance. However, the error is, in general, much smaller for adjacent channel interferers and interferers with different modulation than the desired user. Simulation results confirm the impact of training sequences on joint demodulation. We show simulation results for CCI and ACI performance of joint demodulation (JD) with GMSK and 8- PSK modulations. We find that a large gain is achieved over conventional (single-user) demodulation (CD) in dominant interferer–limited environments. The outline of this paper is as follows. Section II describes the system model. Section III describes various components of the joint demodulation receiver. Numerical results are provided in Section IV, where we first present analytical and simulation results for various training sequence pairs (Section IV-A) and next present simulation results comparing CD and JD receiver performance in CCI and ACI environments (Sections IV-B- IV-D). Finally, conclusions are drawn in Section V. II. SYSTEM MODEL Consider a mobile communication system where the desired signal is received in the presence of a dominant interfering signal, transmitted from a cochannel or adjacent channel base station, and additive noise. The received signal is filtered by a