ANALYSIS OF THE PILOT CONTAMINATION EFFECT IN VERY LARGE MULTICELL MULTIUSER MIMO SYSTEMS FOR PHYSICAL CHANNEL MODELS Hien Quoc Ngo  Thomas L. Marzetta † Erik G. Larsson   Department of Electrical Engineering (ISY), Link ¨ oping University, 581 83 Link¨ oping, Sweden † Bell Laboratories, Alcatel-Lucent, Murray Hill, NJ 07974, USA ABSTRACT We consider multicell multiuser MIMO systems with a very large number of antennas at the base station. We assume that the channel is estimated by using uplink training sequences, and we consider a physical channel model where the angular domain is separated into a finite number of directions. We analyze the so-called pilot con- tamination effect discovered in previous work, and show that this effect persists under the finite-dimensional channel model that we consider. We further derive closed-form bounds on the achievable rate of uplink data transmission with maximum-ratio combining, for a finite and an infinite number of base station antennas. Index Terms— Pilot contamination, very large MIMO systems. 1. INTRODUCTION Multiuser multiple-input multiple-output (MU-MIMO) systems can offer a spatial multiplexing gain without the requirement of multi- ple antennas at the users [1]. Most studies have assumed that the base station has some channel state information (CSI). The problem of not having an a priori CSI at the base station has been considered in [2,3], assuming that the channel estimation is done by using uplink pilots. This requires time-division duplex (TDD) operation. Refer- ences [2, 3] considered a single-cell setting. This is only reasonable when the pilot sequences available in each cell are orthogonal to those in other cells. However, in practical cellular networks, chan- nel coherence times are not long enough to allow for orthogonal- ity between the pilots in different cells. Therefore, non-orthogonal training sequences must be utilized and hence, the multicell setting should be considered. In the multicell scenario with non-orthogonal pilots in different cells, channel estimates obtained in a given cell will be impaired by pilots transmitted by users in other cells. This effect, called “pilot contamination” has been analyzed in [4]. Recently, [5] considered the multicell MU-MIMO system with very large antenna arrays at the base station and showed that when the number of antennas in- creases without bound, uncorrelated noise and fast fading vanish and the pilot contamination effect dictates the ultimate limit on the sys- tem performance. Most the studies referred to above assume that the channels are independent [2–4] or the channel vectors for different users are asymptotically orthogonal [5]. However, in reality, the MIMO channel is generally correlated because the antennas are not suffi- ciently well separated or the propagation environment does not offer This work was supported in part by ELLIIT and the Swedish Research Council (VR). E. G. Larsson is a Royal Swedish Academy of Sciences (KVA) Research Fellow supported by a grant from Knut and Alice Wallenberg Foun- dation. rich enough scattering. In this paper, we investigate the multicell MU-MIMO with large antenna arrays assuming a physical channel model. More precisely, we consider a finite-dimensional channel model in which the angular domain is partitioned into a large, but finite number of directions which is small relative to the number of base station antennas. The channels are estimated by using uplink training (assuming TDD operation, as in previous work). For such channels, the number of parameters to be estimated is fixed regard- less of the number of antennas. We show that the pilot contamination effect persists under the finite-dimensional channel model. Further- more, we derive a closed-form lower bound on the achievable rate of the uplink transmission, assuming maximum-ratio combining at the base station. This bound is valid for a large but finite number of antennas. 2. SYSTEM MODEL Consider L cells, where each cell contains one base station equipped with M antennas and K single-antenna users. Assume that the L base stations share the same frequency band. We consider uplink transmission, where the lth base station receives signals from all users in all cells. See Fig. 1. Then, the M × 1 received vector at the lth base station is given by y l = √ pu L  i=1 Υ il xi + n l (1) where Υ il represents the M ×K channel matrix between the lth base station and the K users in the ith cell, i.e., [Υ il ] m,k is the channel coefficient between the mth antenna of the lth base station and the kth user in the ith cell; √ puxi is the K × 1 transmitted vector of K users in the ith cell (the average power used by each user is pu); and n l contains M × 1 additive white Gaussian noise (AWGN). We assume that the elements of n l are Gaussian distributed with zero mean and unit variance. 2.1. Physical Channel Model Here we introduce the finite-dimensional channel model that is used throughout the paper. The angular domain is divided into a large but finite number of directions P . P is fixed regardless of the number of base station antennas (P<M). Each direction, corresponding to the angle φ k , φ k ∈ [−π/2,π/2], k =1, ..., P , is associated with an M × 1 array steering vector a (φ k ) which is given by a (φ k )= 1 √ P  e −jf 1 (φ k ) ,e −jf 2 (φ k ) , ..., e −jf M (φ k )  T (2) where fi (φ) is some function of φ. The channel vector from kth user in the ith cell to the lth base station is then a linear combination of the 3464 978-1-4577-0539-7/11/$26.00 ©2011 IEEE ICASSP 2011