IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 58, NO. 5, JUNE 2009 2299 Iterative Group-Blind Multiuser Detection and Decoding With Signal Rank Estimation for Coded CDMA Systems Shahram Talakoub, Member, IEEE, and Behnam Shahrrava, Member, IEEE Abstract—In this paper, a turbo receiver structure is proposed for the uplink of coded code-division multiple-access (CDMA) systems in the presence of unknown users. The proposed receiver consists of two stages following each other. The first stage performs soft interference cancellation and group-blind linear minimum mean square error (MMSE) filtering, and the second stage per- forms channel decoding. The proposed group-blind linear MMSE filter suppresses the residual multiple-access interference (MAI) from known users based on the spreading sequences and the channel characteristics of these users while suppressing the in- terference from other unknown users using a subspace-based blind method. The proposed receiver is suitable for suppressing intercell interference in heavily loaded CDMA systems. Since the knowledge of the number of unknown users is crucial for the proposed receiver structure, a novel estimator is also proposed to estimate the number of unknown users in the system by exploiting the statistical properties of the received signal. Simulation results demonstrate that the proposed estimator can provide the number of unknown users with high accuracy; in addition, the proposed group-blind receiver integrated with the new estimator can signif- icantly outperform the conventional turbo multiuser detector in the presence of unknown users. Index Terms—Code-division multiple-access (CDMA) systems, group-blind detector, interference cancellation, rank estimation, turbo multiuser detection. I. I NTRODUCTION I N RECENT years, iterative (turbo) processing techniques have received considerable attention, inspired by the discov- ery of turbo coding [1]. Exhaustive research in related topics produced different algorithms that follow the turbo decoding approach to provide a similar gain in performance. Turbo multiuser detection applies the same principle, namely, the iterative exchange of soft information among different blocks to improve the system performance [2]. In such detectors, channel decoding is combined with multiuser detection in a way that the outputs of channel decoders are fed back to the multiuser detec- tor to iteratively improve the system performance. The optimal turbo multiuser detector (TMUD) by exploiting the maximum a posteriori (MAP) algorithm for detection and decoding was Manuscript received January 10, 2008; revised July 14, 2008. First published November 21, 2008; current version published May 11, 2009. The review of this paper was coordinated by Prof. X. Xia. S. Talakoub is with Research in Motion (RIM) Ltd., Waterloo, ON N2L 3L3, Canada (e-mail: stalakoub@rim.com). B. Shahrrava is with the Department of Electrical and Computer Engi- neering, University of Windsor, Windsor, ON N9B 3P4, Canada (e-mail: shahrrav@uwindsor.ca). Digital Object Identifier 10.1109/TVT.2008.2009940 proposed in [3]. In the same paper, a less complex approach based on interference cancellation and linear minimum mean square error (MMSE) filtering was proposed. In the uplink of code-division multiple-access (CDMA) sys- tems, group-blind detection algorithms can be employed since the base station has knowledge of the spreading sequences of a group of users within its own cell, while it is blind with respect to other users from adjacent cells. In [4]–[6], various types of linear and nonlinear detection schemes have been proposed for uncoded synchronous CDMA systems. In [7], the idea in [5] and [6] has been generalized, and several forms of group-blind detectors for uncoded CDMA based on different criteria have been developed. For coded CDMA systems, in [8], a TMUD was proposed for the uplink of asynchronous CDMA systems in the presence of unknown users. To suppress both intercell and intracell inter- ference, the detector performed soft interference cancellation for each user, in which estimates of the multiuser interference from the other known users and an estimate for the interference caused by unknown users are subtracted from the received signal. However, in [13], mathematically, we have proved that such a multiuser detector fails to converge when the a priori information provided by the single-user decoders, at the input of the detector, is large enough (the proof is also given in the Appendix). This situation may happen at both high and low signal-to-noise ratios (SNRs) after only a few iterations when the single-user decoders converge to the right or wrong code words. In [14], the multiple-access interference (MAI) caused by unknown users was approximated as Gaussian noise. The method tried to improve the performance of the TMUD by employing the covariance of this noise and the ambient channel noise at the receiver. Since the bit-error-rate (BER) performance of most multiuser detectors are almost insensi- tive to any mismatch between the actual noise variance and its estimated value within a certain range of the noise level [15], such a detector cannot provide considerable performance improvement. All group-blind algorithms based on subspace techniques, for both coded and uncoded CDMA, require the number of known and unknown users. It is reasonable to expect to obtain the number of known users from higher layers (i.e., network layers), but it is very unrealistic to expect to have such infor- mation regarding unknown users in the system. A likelihood approach to estimate the number of users based on the Akaike information criterion (AIC) and minimum description length (MDL) criterion was proposed in [10]. These criteria mostly 0018-9545/$25.00 © 2009 IEEE