1880 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 52, NO. 11, NOVEMBER 2004 Near-Capacity Coding in Multicarrier Modulation Systems Masoud Ardakani, Student Member, IEEE, Tooraj Esmailian, and Frank R. Kschischang, Senior Member, IEEE Abstract—We apply irregular low-density parity-check (LDPC) codes to the design of multilevel coded quadrature amplitude modulation (QAM) schemes for application in discrete multitone systems in frequency-selective channels. A combined Gray/Unger- boeck scheme is used to label each QAM constellation. The Gray-labeled bits are protected using an irregular LDPC code with iterative soft-decision decoding, while other bits are protected using a high-rate Reed–Solomon code with hard-decision decoding (or are left uncoded). The rate of the LDPC code is selected by analyzing the capacity of the channel seen by the Gray-labeled bits and is made adaptive by selective concatenation with an inner repetition code. Using a practical bit-loading algorithm, we apply this coding scheme to an ensemble of frequency-selective channels with Gaussian noise. Over a large number of channel realizations, this coding scheme provides an average effective coding gain of more than 7.5 dB at a bit-error rate of 10 and a block length of approximately 10 b. This represents a gap of approximately 2.3 dB from the Shannon limit of the additive white Gaussian noise channel, which could be closed to within 0.8–1.2 dB using constellation shaping. Index Terms—Discrete multitone systems, low-density parity-check (LDPC) codes, multilevel-coded modulation, fre- quency-selective channels. I. INTRODUCTION I N DISCRETE multitone (DMT) systems, there are many sub-channels with different signal-to-noise ratios (SNRs). Designing a coding system that meets the requirements of these channels has drawn much attention [1]–[3]. In standard asymmetric digital subscriber lines, a trellis-coded-modulation scheme in concatenation with a Reed–Solomon code is used [4], providing approximately 5-dB coding gain at a symbol-error rate (SER) of 10 . The two principal classes of codes for the high-SNR regime are lattice codes and trellis codes. The SNR gap between un- coded baseline performance of quadrature-amplitude modula- tion (QAM) and Shannon limit is 9 dB at an SER of 10 . At this SER, the coding gain of the Leech lattice in dimension 24 is less than 4 dB [5]. With a 512-state trellis code, an effective Paper approved by W. E. Ryan, the Editor for Modulation, Coding, and Equal- ization of the IEEE Communications Society. Manuscript received June 19, 2003; revised March 3, 2004 and May 26, 2004. M. Ardakani and F. R. Kschischang are with the Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON M5S 3G4, Canada (e-mail: masoud@comm.utoronto.ca; frank@comm.utoronto.ca). T. Esmailian was with the Edward S. Rogers Sr. Department of Elec- trical and Computer Engineering, University of Toronto, Toronto, ON M5S 3G4, Canada. He is now with the Research and Development Group, Edge- water Computer Systems, Inc., Kanata, ON K2K 3G6, Canada (e-mail: tooraj@eecg.utoronto.ca). Digital Object Identifier 10.1109/TCOMM.2004.836560 coding gain of 5.5 dB can be achieved, but it seems that ap- proaching a coding gain close to 6 dB with trellis codes is very difficult [5]. All of the coding gains that we refer to are only due to coding. Using proper shaping techniques, a shaping gain of up to 1.53 dB can be added independent of the coding gain. However, even with a 512-state trellis code and achieving the ultimate shaping gain, there is a gap of 2 dB from the Shannon limit. The goal of this paper is to provide near-capacity coding techniques for the high-SNR regime and more generally for DMT systems. At low SNR, the problem of near-capacity coding has been studied extensively, and it has been shown [6]–[9] that turbo codes and low-density parity-check (LDPC) codes can approach the capacity of many channels with prac- tical complexity. For high-SNR channels, however, multilevel modulation is required. The main problem of using multilevel symbols in these codes is that large alphabet sizes create prohib- itively large decoding complexity. To overcome this problem, one can use multilevel coding, which allows one to apply binary codes to multilevel modulation schemes. However, in DMT systems, dealing with subchannels whose SNR is dif- ferent and use different constellation size is a challenge. In [10], a regular high-rate LDPC code is used for error cor- rection in a digital subscriber line (DSL) transmission system. The maximum reported coding gain at a bit-error rate (BER) of 10 is 6.2 dB, which, compared to the maximum possible coding gain (8.3 dB for the additive white Gaussian noise (AWGN) channel), shows a gap of more than 2 dB from the Shannon limit. The goal of [10] is to provide a coding system for DSL transmission systems with practical complexity and suitable structure for hardware implementation, so highly efficient irregular codes, which are more difficult to implement, were not considered. The idea of using LDPC codes together with coded modulation is used in [11] as well. In [2], a turbo code has been used for ADSL and a coding gain of 6 dB at an SER of (equivalent to approximately 6.8 dB at an SER of ) is reported. This amounts to a 1.5-dB gap from the Shannon limit for the AWGN channel. In this paper, we address the problem of coding for DMT systems and propose a coding scheme based on a combina- tion of irregular LDPC codes and multilevel coding, which pro- vides an average coding gain of more than 7.5 dB at a message error rate of 10 . This is equivalent to a gap of approximately 0.8 dB from the Shannon limit for the AWGN channel. The de- coding complexity in our system is comparable with a 512-state trellis code. The main difference between our approach and other work on use of turbo codes/LDPC codes for DMT systems is that our 0090-6778/04$20.00 © 2004 IEEE