3864 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 62, NO. 11, NOVEMBER 2014 Lattice-Reduction-Aided Conditional Detection for MIMO Systems Hossein Najafi, Member, IEEE, and Mohamed Oussama Damen, Senior Member, IEEE Abstract—We introduce a low-complexity detector with near- optimal performance for transmission over multi-antenna systems. By using lattice basis reduction for generating almost orthog- onal channel submatrices, we enhance the conditional opti- mization technique to implement a fast yet efficient detector. The lattice-reduction-aided (LRA) conditional method is presented as a general detection technique over fading channels to yield sig- nificant saving in computational complexity while achieving close to Maximum Likelihood (ML) error performance. By employing the orthogonality defect factor as a universal measure to select a near-orthogonal channel submatrix for conditional detection, we implement efficient detectors for MIMO systems. In particular, an almost optimal decoder with linear complexity for the Golden code is presented over quasi-static channels. Index Terms—Conditional detection, golden code, lattice re- duction, low-complexity detection, maximum likelihood detec- tion, multiple-input multiple-output (MIMO) systems, space-time codes. I. I NTRODUCTION F INDING fast and efficient decoding methods for space- time codes and high-rate MIMO transmission is an im- portant design problem in wireless communications. Decoders with low computational complexity but yet, close to the optimal performance are challenging issues for practical implementa- tion of schemes with multiple transmit and receive antennas such as the V-BLAST (Vertical Bell Labs Layered Space-Time) transmission model [1] and the Golden code [2], [3]. The low- complexity and powerful decoders are of special interest since they have many applications for widely incorporated MIMO schemes in the wireless standards such as 3GPP LTE, IEEE 802.16 WiMAX and IEEE 802.11 WLANs. Motivated by ef- ficient search algorithms in the lattice theory, the multi-antenna detection problem over fading channels can be translated into finding the closest point in the lattice formed by the channel ma- trix [4]–[7]. Lattice basis reduction, such as the LLL algorithm [8] as an efficient one, is widely used for the implementation of the search algorithms. In a different approach, by taking advantage of orthogonal sub-channels, a low-complexity de- Manuscript received January 8, 2014; revised May 11, 2014 and August 5, 2014; accepted September 12, 2014. Date of publication February 10, 2014; date of current version November 18, 2014. The associate editor coordinating the review of this paper and approving it for publication was M. Matthaiou. The authors are with the Department of Electrical and Computer Engineer- ing, University of Waterloo, Waterloo, ON N2L 3G1, Canada (e-mail: hnajafi@ uwaterloo.ca; mdamen@uwaterloo.ca). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TCOMM.2014.2361337 coder for multiplexed designs was proposed in [9] where it was shown that optimal decoding can be implemented by employing the conditional optimization technique [10]. In [11], using the conditional decoder together with selecting the best submatrix choice, a fast decoder with quadratic complexity in the size of symbol constellation with essentially ML performance was proposed for the Golden code. Low-complexity and almost ML performance of conditional decoding has also been studied in [12] for several space-time block codes (STBC). In this work, we propose a fast universal detector 1 for multi- antenna systems by employing lattice reduction as a major part in the conditional optimization. By taking advantage of the near-orthogonal matrix at the LLL output, close to optimal performance is achieved over fading channels with significant saving in complexity. For the fast low-complexity implementa- tion of the conditional detector, especially for a small dimension of the second subchannel, the use of lattice reduction and effective selection of channel submatrices are vital for the near-ML performance. To select the best available channel submatrix, we use the orthogonality defect factor as a general measure that perfectly fits the orthogonality requirement of the conditional optimization. For a given dimension of the conditioned symbols, we compare the orthogonality defect factor of the LLL output of the possible choices and select the more orthogonal submatrix to implement the conditional detection on it. We also discuss the use of more powerful but still simple detection methods such as the decision-feedback- equalizer (DFE) at the first part of decoding, instead of the zero- forcing (ZF), to improve the overall detection and close the gap to the ML solution. We apply the new detectors for three scenarios over quasi- static channels: the Golden code, the diagonal algebraic space- time (DAST) block codes [13], [14] and the spatial multiplexing MIMO system (or the V-BLAST transmission model where we send independent symbols over different transmit antennas). For a fast detection, we aim to implement the lattice-reduction- aided (LRA) conditional detector with O(N ) complexity, where N is the size of the employed signal constellation. Therefore, we maximize the likelihood function for a single constellation point conditioned on the rest of points which are already estimated by the LRA methods. The full-diversity near- ML performance is studied and also verified by the numerical results. Additionally, various detectors with close to ML error performances are presented for a larger MIMO system. Finally, 1 We use the terms “detection” and “decoding” interchangeably throughout the paper. 0090-6778 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.