PROTOTYPE WAVEFORM OPTIMIZATION FOR AN OFDM/OQAM SYSTEM WITH HEXAGONAL TIME-FREQUENCY LATTICE STRUCTURE Mohamed Siala * and Abbas Yongaçoglu † * SUP’COM, Cité Technologique des Communications, Route de Raoued Km 3,5, 2083 Ariana, Tunisia Email : mohamed.siala@supcom.rnu.tn † School of Information Technology and Engineering, University of Ottawa, K1N 6N5, Ontario, Canada Email: yongacog@site.uottawa.ca ABSTRACT This paper deals with the optimization of the time- frequency localization of the prototype waveform for OFDM/OQAM with hexagonal time-frequency lattice. The set of orthonormal waveforms among which the optimally localized one should be searched are obtained as the orthonormalization of mother functions well localized in time-frequency. To preserve as much as possible the good localization of the original mother functions, the canonical form of the orthonormalization procedure is used. For an easy and systematic way to generate well localized mother functions, we use the orthonormal base of Hermite functions, which are known to provide, in decreasing order, the best time-frequency localization. To preserve the /3 π rotational invariance of the hexagonal lattice in the ambiguity function of each candidate prototype function, only Hermite functions with indices multiple of 6 are allowed. The obtained numerical results show that the optimized prototype waveform outperforms in localization the best localized waveform for OFDM/OQAM with square time-frequency lattice. I. INTRODUCTION With ADSL, DVB-T and WLAN 802.11a, conventional OFDM is becoming the de facto technique for reliable communication over time-dispersive channels [1,2,3]. Unfortunately, this technique is not suited to frequency dispersive channels, due to bad frequency localization of its prototype function. An alternative, to overcome this drawback, is to use OFDM/OQAM. As evidenced by the Balian-Low theorem and the time and frequency exponential decrease of the IOTA function [4], only OFDM/OQAM allows very well localized prototype function to be used. In [5], we have proposed a new version of OFDM/OQAM, using a hexagonal time-frequency lattice instead of the conventional square time-frequency lattice. As pointed out in [6], the hexagonal lattice offers the best packing radius in the two-dimensional space and hence alleviates the time- frequency localization loss due to the orthonormal constraints put on the prototype waveform. This fact has been confirmed in [5], where we have shown that the prototype waveform resulting from the orthonormalization of the Gaussian function, has better localization when the hexagonal lattice is used instead of the square lattice. In [7], we have searched for the best localized orthonormal waveform for the conventional square lattice and disproved the conjecture that the IOTA function, resulting from the orthonormalization of the Gaussian function, is the most localized prototype function. Following the philosophy of [7], our aim in this paper is to determine the most localized waveform for the hexagonal lattice, hoping that the corresponding localization will be the utmost limit one can ever reach. The paper is organized as follows. In Section II, we make a full characterization of OFDM/OQAM systems with hexagonal time-frequency lattices. In Section III, we specify the orthonormality criterion for the OFDM/OQAM system. Section IV stresses the equivalence of this criterion with that of OFDM with ½ hexagonal lattice density. In Section V, we present a general form of the orthonormalization procedure for the construction of orthonormal prototype functions from arbitrary mother functions and underline the merits of its canonical version. In Section VI, we derive an appropriate representation of candidate mother functions which guarantee good time-frequency localization as well as /3 π rotational invariance of the prototype function inherited from the /3 π rotational invariance of the hexagonal lattice. The numerical results are given in Section VII and the conclusion is reached in Section VIII. II. SYSTEM CHARACTERISTICS OFDM/OQAM systems are based on square time-frequency lattices, with density 2 ∆= , in order to guarantee the same spectral efficiency as guard interval-free OFDM systems. The hexagonal time-frequency lattice specifying the time- frequency shifts of the prototype function to be used in the generation of the modulated signal is specified by the generating matrix 1 1/2 0 3/2 ρ   =       G , (1) where 4 2/ 3 1/ 3 ρ = ∆= . Any other rotated version of this hexagonal lattice could also be considered and dealt with as well. The adopted time-frequency hexagonal lattice is therefore composed of time-frequency points of the form ( , ) ( / 2, 3 / 2) mn mn m n n ν τ ρ = + , (2) where mn ν and mn τ denote respectively frequency and time, and m and n are arbitrary integers. For each point in the hexagonal time-frequency lattice, we associate a time- frequency shifted version the parity prototype waveform, () t φ , of the form () exp( )( )exp( 2 ) mn mn mn mn t j t j t φ θ φ τ πν = - , (3) where mn θ are arbitrary phase parameters. As shown in [5], the prototype function () t φ should be even or odd and the phase parameters mn θ must be given by 0 for and even /4 for even and odd /2 for odd and even 3 /4 for and odd mn m n m n m n m n π θ π π    =     (4) 1-4244-0779-6/07/$20.00 ©2007 IEEE