Capacity Analysis of Correlated Multiple Antenna Systems With Finite Rate Feedback Jun Zheng and Bhaskar D. Rao University of California at San Diego La Jolla, CA 92093–0403 Email: juzheng@ucsd.edu, brao@ece.ucsd.edu Abstract— We consider in this paper the analysis of transmit beamforming methods in multiple antenna systems over corre- lated fading channels and with finite rate feedback of the channel state information. The problem is formulated as a general vector quantization problem with encoder side information, constrained quantization space and non-mean-square distortion function. By utilizing the high-resolution distortion analysis of the generalized quantizer, which is applicable to a wide range of scenarios, we obtain a tight lower bound on the capacity loss of the finite rate quantized MISO system over correlated fading channels. The lower bound of the capacity loss of correlated MISO channels is a generalization of existing results available for i.i.d. channels. The bound, in addition to providing insight into the exact nature of dependence of the quantization loss on the channel correlation matrix, indicates that the loss is less than that of the i.i.d. channels but with the same exponential decaying factor w.r.t. the feedback rate. The generality of the framework is further demonstrated by considering its application to the analysis of suboptimal mismatched channel quantizers, i.e. quantizers designed with an incorrect channel covariance matrix, and comparing it to systems with optimal quantizers. Finally, numerical and simulation results of the finite rate quantized MISO beamforming system with codebook designed by the Lloyd algorithm are presented that confirm the accuracy of the obtained analytical results. I. I NTRODUCTION Communication systems using multiple antennas at both the transmitter and the receiver have recently received much attention due to their promise of providing significant capacity increases in a wireless fading environment. The performance of the multiple antenna systems depends heavily on the availability of the channel state information (CSI) at the transmitter (CSIT) and at the receiver (CSIR). Most of the MIMO system design and analysis adopt one of two extreme CSIT assumptions, complete CSIT and no CSIT. In this paper, we consider systems with CSI assumptions in between these extremes. We assume perfect CSIR is available at the receiver, and focus our attention on MIMO systems where CSI is conveyed from the receiver to the transmitter through a finite rate feedback link. Recently, several interesting papers have appeared, proposing design algorithms as well as analytically quantifying the performance of the finite rate feedback multi- ple antenna systems [1] - [8]. This paper attempts to add to this body of knowledge. Narula et. al. considered in [1] a multiple transmit antennas and single receive antenna (MISO) system which employs finite-rate feedback to describe the beamforming vector. The Lloyd algorithm [2] was utilized to designing the optimum beamforming vector codebook, where both the channel gain This research was supported in part by CoRe grant No.02-10109 sponsored by Ericsson and in part by the U. S. Army Research Office under the Multi- University Research Initiative (MURI) grant # W911NF-04-1-0224. and the system mutual information were used as performance metrics. Based on the geometrical properties of the channel space, Mukkavilli et. al. [3] derived a universal lower bound on the outage probability of quantized MISO beamforming systems with arbitrary number of transmit antennas t over i.i.d. Rayleigh fading channels. Love et. al. [4] proposed a codebook design criterion based on minimizing the maximum inner product of the beamforming vectors in the codebook, and related the min-max problem to that of Grassmannian line packing which is the problem of maximally separating lines in the Grassmann manifold. The authors also investigated in [5] the problem of quantizing the beamforming vector under a per-antenna power constraint, which is named as quantized equal gain transmission. This problem was recently revisited by Murthy et. al. in [6] and a closed form capacity loss analysis was obtained. Vector quantization (VQ) techniques combined with Lloyd algorithm was utilized by Xia et. al. in [7] and Roh et. al. in [8]. The authors derived an (weighted) inner product criterion and used the Lloyd algorithm [2] to generate the codebook that specifically optimize for both the statistical distribution of the vector (or matrix) channel as well as the specific performance metric (for example, the mutual information rate). Both of these groups analyzed the performance of MISO systems with limited-rate feedback in the case of i.i.d. Rayleigh fading channels, and obtained closed form expressions of the capacity loss (or SNR loss) in terms of feedback rate B and antenna size t. In this paper, we consider the analysis of transmit beam- forming methods in multiple antenna systems over correlated fading channels and with finite rate CSI feedback. The problem is formulated as a general vector quantization problem with encoder side information, constrained quantization space and non-mean-square distortion function. By utilizing the high- resolution distortion analysis of the generalized quantizer pro- vided in [9], which is applicable to a wide range of scenarios, we obtain a tight lower bound on the capacity loss of the finite rate quantized MISO systems over correlated fading channels. The lower bound of the capacity loss of correlated MISO channels are a generalization of existing results available on i.i.d. channels, and its approximations in high-SNR regimes are also provided. The bound provides insight into the exact nature of dependence of the quantization loss on the channel correlation matrix, which indicates that the loss is less than that of the i.i.d. channels but with the same exponential decaying factor w.r.t. the feedback rate. The generality of the frame- work is further demonstrated by considering its application to the analysis of suboptimal mismatched channel quantizers, i.e. quantizers designed with an incorrect channel covariance