IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 50, NO. 1, JANUARY 2002 65 Layered Space–Time Receivers for Frequency-Selective Wireless Channels Angel Lozano, Member, IEEE, and Constantinos Papadias, Member, IEEE Abstract—Recent results in information theory have demon- strated the enormous potential of wireless communication systems with antenna arrays at both the transmitter and receiver. To exploit this potential, a number of layered space–time architectures have been proposed. These layered space–time systems transmit parallel data streams, simultaneously and on the same frequency, in a multiple-input multiple-output fashion. With rich multipath prop- agation, these different streams can be separated at the receiver because of their distinct spatial signatures. However, the analysis of these techniques presented thus far had mostly been strictly narrowband. In order to enable high-data-rate applications, it might be necessary to utilize signals whose bandwidth exceeds the coherence bandwidth of the channel, which brings in the issue of frequency selectivity. In this paper, we present a class of layered space–time receivers devised for frequency-selective channels. These new receivers, which offer various performance and com- plexity tradeoffs, are compared and evaluated in the context of a typical urban channel with excellent results. Index Terms—Adaptive antennas, BLAST, equalization, fre- quency selectivity, interference cancellation, layered architectures, MIMO, space–time processing. I. INTRODUCTION R ECENT information theory results have shown the enormous spectral efficiency potential of wireless com- munication systems with antenna arrays at both the transmitter and receiver, in particular when the channel and array structures are such that the transfer functions between different transmit and receive antenna pairs are sufficiently uncorrelated [1]–[4]. To exploit this potential, a number of layered space–time (BLAST) architectures have been proposed [5], [6]. BLAST systems transmit parallel data streams, using multiple antennas, simultaneously and in the same frequency band. With rich multipath propagation, these different streams can be separated at the receiver because of their distinct spatial signatures. Remarkably, in its original form, BLAST does not require the transmitter to possess any channel information; only the receiver is required to estimate the channel. Nonetheless, provided the scattering richness is sufficiently high, the spectral efficiency attainable—in this open-loop form—is often very close to the spectral efficiency supported with full channel information at the transmitter [7]. In this paper, we focus our attention on such rich-scattering environments where BLAST performs at its best. Paper approved by A. Ahlen, the Editor for Modulation and Signal Design of the IEEE Communications Society. Manuscript received March 15, 2000; revised October 15, 2000, and May 15, 2001. The authors are with Bell Laboratories (Lucent Technologies), Crawford Hill Laboratory, Holmdel, NJ 07733 USA (e-mail: aloz@lucent.com; papadias@lu- cent.com). Publisher Item Identifier S 0090-6778(02)00512-3. A form of BLAST that is very attractive for its relative simplicity was introduced in [6], [8] and labeled as Vertical BLAST (V-BLAST). In V-BLAST, every transmit antenna radiates an equal-rate independently encoded stream of data. This transmit structure enables the utilization, at the receiver, of interference rejection and cancellation techniques [9], [10] with the added advantage that the multiple streams are precisely synchronized. A V-BLAST receiver can be regarded, therefore, as a synchronous multiuser detector with ordered successive cancellation. This type of successive cancellation method has already proved very effective in other contexts [11]–[13]. Nonetheless, the V-BLAST formulation and analysis presented thus far had been strictly narrowband. In order to extend that formulation to the more general case of frequency-selective channels, two dual approaches exist, namely, orthogonal frequency-division multiplexing (OFDM) or high-speed serial equalization. Whereas the first approach was the one chosen in [3], in this paper we concentrate on the high-speed serial case, where the receiver adopts—in its full generality—the form of a multiple-input multiple-output (MIMO) decision-feedback equalizer (DFE). Linear MIMO equalizer alternatives are also included in this framework simply by setting the length of the DFE feedback section to zero. MIMO equalizers present a significant challenge because of the need to detect signals buried in both co-channel inter- ference (CCI) as well as inter-symbol interference (ISI), in addition to noise. They were studied in the past in the context of cross-coupled channels and dually polarized radio systems among other problems [14]–[16]. With the exploding interest in space–time processing and multiuser detection in recent years, MIMO DFEs have again attracted significant attention. The optimal settings—in the minimum mean-square error (MMSE) sense—for the MIMO DFE were derived in [17], [18] within the framework of code-division multiple access (CDMA) and in [19], [20] for the case of space-division multiple access (SDMA). However, the settings derived therein correspond, in general, to infinite-length noncausal filters. Furthermore, in some of those analyses, the number of inputs and outputs were constrained to be identical. In our case, the number of transmit and receive antennas need not be the same, so this requirement is dropped. Also, we are interested in solutions that correspond to realizable finite-impulse-response (FIR) filter [21]. Those optimal settings, again in the MMSE sense, have recently been reported in [22]–[24] for a receiver without successive cancellation. In that receiver, only decisions on temporally preceding symbols are fed back into the detection process of each stream [Fig. 1(a)]. CCI contributions from undetected future and on-time symbols are, therefore, not canceled. In this 0090–6778/02$17.00 © 2002 IEEE