IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER 1999 1777 Constrained Coding for Binary Channels with High Intersymbol Interference Razmik Karabed, Paul H. Siegel, Fellow, IEEE, and Emina Soljanin, Member, IEEE Abstract—Partial-response (PR) signaling is used to model com- munications channels with intersymbol interference (ISI) such as the magnetic-recording channel and the copper-wire channel for digital subscriber lines. Coding for improving noise immunity in higher order partial-response channels, such as the “extended” class-4 channels denoted EPR4, E PR4, E PR4, has become an important subject as the linear densities in magnetic recording approach those at which these partial-response channels are the best models of real channels. In this paper, we consider partial- response channels for which ISI is so severe that the channels fail to achieve the matched-filter bound (MFB) for symbol error rate, assuming maximum-likelihood decoding. We show that their performance can be improved to the MFB by high-rate codes based on constrained systems, some of which may even simplify the Viterbi detectors relative to the uncoded channels. We present several examples of high-rate constrained codes for E PR4 and E PR4 channels and evaluate their performance by simulation. Index Terms—Constrained coding, magnetic recording, partial response. I. INTRODUCTION P ARTIAL response (PR) signaling is used to model com- munications channels with intersymbol interference (ISI) such as the magnetic-recording channel and the copper-wire channel for digital subscriber lines. This paper is mainly concerned with high-density, binary-input, magnetic-recording systems. Partial-response channels with the transfer function of the form , have been shown to closely match magnetic-recording channels for a range of linear recording densities [59]. Detectors for these channels, known as PRML detectors, employ PR equalization followed by maximum-likelihood (ML) sequence detection matched to the equalized channel. Most of the original gen- eration of commercially available PRML detectors employed equalization to the PR4 channel and the Viterbi ML detection matched to this channel with additive white Gaussian noise [63], [56]. At higher linear densities, the channel transfer function changes and the PR4 equalization alters the spectral density of the noise by enhancing the high-frequency compo- nents. It has been shown, analytically [51] and by simulation Manuscript received March 29, 1997; revised February 18, 1999. The work of P. H. Siegel was supported in part by the National Science Foundation under Grant NCR-9612802. R. Karabed is with the Siemens Microelectronics, Scotts Valley, CA 95066 USA. P. H. Siegel is with the University of California at San Diego, La Jolla, CA 92093 USA. E. Soljanin is with the Mathematical Sciences Research Center, Bell Labs, Lucent Technologies, Murray Hill, NJ 07974 USA. Communicated by N. Seshadri, Associate Editor for Coding Techniques. Publisher Item Identifier S 0018-9448(99)05882-4. [47], that this in turn causes a loss in performance of the Viterbi detector, which is an optimal algorithm in the case of white noise. This loss is referred to as the equalization loss. To reduce the equalization loss, a PR polynomial that more closely matches the channel should be chosen as the equalization target. For current linear recording densities the appropriate choice is or EPR4, and at higher densities or E PR4 becomes better. Future systems may incorporate E PR4 equalization, which corresponds to This paper is particularly concerned with systems using E PR4 and E PR4. The general problem of coding for binary-input constrained PR channels has been approached in several ways. One approach, initiated by Wolf and Ungerboeck in [66], uses error- control codes with good Hamming distance on a precoded channel. (A related technique, employing a channel post-coder, was presented by Calderbank, Heegard, and Lee [14].) This approach was justified by showing that the minimum squared- Euclidean distance of a code on a precoded ISI channel of the form is lower-bounded by its minimum Hamming distance. Codes with good Hamming distance are known and can be found in numerous tables available in the literature, as for example in Lin and Costello [30, p. 331]. Hole [20] and Hole and Ytrehus [21] have refined this technique and found codes with improved properties and reduced detector complexity for the precoded PR4 channel. A second approach, as described by Karabed and Siegel in [25], is to use codes with spectral nulls at the frequencies of the spectral nulls of the channel. This design approach was justified by showing that matching of the code and the channel spectral nulls provides a substantial increase in the minimum Euclidean distance for an important class of partial response channels. These codes are known as matched spectral null (MSN) codes. Several approaches were proposed for simplified decoding of MSN codes on the PR4 channel. Knudson et al. [28] introduced a form of concatenated detection for MSN-coded PR4, sometimes referred to as post-processing. (A related form of post-processing was applied to EPR4 detection by Wood [68] and Knudson [16].) Fredrickson et al. [18] and Mittel- holzer et al. [37] developed MSN codes and related codes with decoders characterized by time-varying trellis structures. Another approach to simplification of MSN encoders and decoders on the PR4 channel was also recently proposed by Soljanin in [53] and [54]. Based upon these techniques, a number of proposals for performance-improving codes for the PR4 channel have been 0018–9448/99$10.00  1999 IEEE