Proceedings of 14th International Conference on Computer and Information Technology (ICCIT 2011) 22-24 December, 2011, Dhaka, Bangladesh Compensation of Quadrature Imbalance in an Optical 16-QAM Digital Coherent Receiver Md. Saifuddin Faruk Dept. of Electrical and Electronic Engg., Dhaka University of Engineering and Technology, Gazipur, Bangladesh faruk@duet.ac.bd Abstract In this paper, the impact of quadrature imbalance (QI) is analyzed for an optical 16-quadrature amplitude modulation (16-QA) transmission system that employs digital coherent receivers. A compensation method for Ql in digital domain is demonstrated using four real-valuedilters connected in butterly coigurations and adapted by decision directed least-mean-square (DD-L) algorithm. The efectiveness of the proposed Ql compensation scheme is veried by numerical simulations. I. INTRODUCTION Digital coherent optical receIvers enable demodulation of higher-order complex modulation formats [I] . Thel6-quadrature amplitude modulation (16-QAM) is one of the popular choices for such receivers due to its high spectral eiciency. Ideally, the in-phase () and quadrature (Q) components of the complex modulated signal should be orthogonal. However, such orthogonality can be destroyed in the coherent optical communication system for various reasons such as incorrect bias point setting of the modulator, imperfection of optical 90 ° hybrid used in phase-diversity receivers, photodiode responsivity mismatch etc. These imperfections of devices introduce amplitude and phase mismatch between I and Q signals and termed as quadrature imbalance (QI). Several methods of QI compensation have been reported so far [2-5]. In [2] Gram-Schmidt orthogonalization procedure (GSOP) is used for QI compensation of quadrature phase shit keying (QPSK) modulated signal; however such algorithm complexity increases in 16-QAM system as a long averaging span is required. In [3] QI is compensated by parabolic itting method on the constellation of received signals which cannot be applied effectively for 16-QAM modulation format, especially at low optical signal-to-noise ratio (OSNR). In [4, 5] a dedicated constrained QI equalizer based on constant modulus algorithm (CMA) is proposed for QPSK modulated system. However, the CMA cannot perform well for 16-QAM. All the methods mentioned previously use a dedicated DSP circuit for QI compensation and none of them analyze the impact of QI considering 16- QAM format. In this paper, we propose to use four adaptive ilters with real-valued coeicients connected in two-by-two butterly coniguration instead conventionally used one complex-valued ilter. With such a coniguration, apart rom its usual unctionalities, the adaptive ilters can compensate for QI effectively. For the ilter adaptation, the decision directed least-mean-square (DD-LMS) algorithm is chosen as it is more eicient compare to the commonly used CMA for 16-QAM transmission system. The proposed method is veriied by intensive computer simulations. II. BACKGROUND THEORY In the absence of QI (ideal case), the photodetector current that convert the output optical signals of 90° hybrid at the receiver to the electrical signals, are given as �� AcosJ �� AsinJ (1) (2) where A is the amplitude of complex modulated signal and 8 is the common phase that includes modulation phase, intermediate requency (IF) offset between transmitting laser and local oscillator (La), laser phase noise etc. On the other hand, in the presence of QI, the received photocurents can be written as � A�osJcos� sinJsin� � l�os� �in J Iv �JAsin��� �J A�iN coN� coN siN� � W co@� I � si@ (3) (4) where the angle E+O=p is the conjugate misalignment which is the offset rom 90°, and a is amplitude mismatch factor. From Eqs. (3) and (4) we can write (5) where superscript T is the transpose operator and the matrix M represents for QI which can be expressed as � cosJ �sin J� M � � siN J co@ � (6) 987·161284-908-9111/$26.00 © 2011 IEEE