XXX SIMPÓSIO BRASILEIRO DE TELECOMUNICAÇÕES – SBrT’12, 13-16 DE SETEMBRO DE 2012, BRASÍLIA, DF Nonlinear Effects Compensation Analysis for Dual Polarization 16QAM Optical Coherent Systems Victor E. S. Parahyba 1* , Eduardo S. Rosa 1 , Júlio C. M. Diniz 1 , Vitor B. Ribeiro 1 and Júlio C. R. F. Oliveira 1 (*)victorp@cpqd.com.br. 1- CPqD Foundation Abstract — High order modulation nonlinear effects has been appointed as the main limitation in coherent optical fiber transmission. Digital back-propagation algorithms are one of the current studied methods to cope with such impairment and extend the systems maximum reach. In this article, we analyzed the digital back-propagation performance in a 224 Gb/s dual polarization 16QAM optical coherent system. It was observed a 35% increase in maximum reached distance and an OSNR gain of 2.6 dB. In order to reduce the huge required computational complexity, an modified back-propagation algorithm is analyzed. Keywords — Nonlinear Effects Compensation, Dual Polarization, 16QAM, Digital signal processing, Optical Coherent Systems I. INTRODUCTION The growing data traffic on telecommunication systems, pushed by broadband internet applications, has made a significant impact on all kinds of communication networks. In order to comply this increasing demand, the bit rate per channel increased from 2.5 Gb/s to 40 Gb/s within ten years, and now 112 Gb/s systems start being deployed. At the same time, in order to fit more capacity into the Wavelength Division Multiplexing (WDM) spectral grid, the optical channel spacing was reduced from 100 to 50 GHz, creating denser optical systems. Due to this new scenario, it is imperative to replace the traditional On-Off Keying (OOK) modulation, present in the vast majority of deployed optical systems, by modulation formats with higher spectral efficiency. In this way, there is intensive research on multilevel modulation formats, like QPSK and QAM, using Polarization Division Multiplexing (PDM) in ultra-long-haul optical transmission systems. PDM-QSPK has been proposed as the standard modulation format for 112 Gb/s optical systems [1]. Such sophisticated transmission scheme requires a coherent receiver structure that preserves all the information of optical field (amplitude, phase and polarization), what can be explored by digital signal processing (DSP) techniques to compensate most of transmission impairments [2]. Recent advances in digital electronics enabled DSP algorithms utilization for digital compensation of various transmission impairments leading to some major achievements in distance and channel capacity [3]. After the efforts made to compensate major linear system impairments such as polarization mode dispersion (PMD) [4] and chromatic dispersion (CD) [5], the next limit in ultra-long-haul transmission are the nonlinear effects imposed by the optical fiber. Digital Back-propagation (DBP) algorithm has become a prominent nonlinear impairment compensation method [6],[7]. DBP implementation consists on solving the reverse nonlinear Schrödinger propagation equation. However, computational complexity of DBP makes it currently unfeasible, so much effort are being made to simplify the DBP algorithm [8],[9]. In this paper, we investigate trough numerical simulation a modified DBP algorithm based on the correlation of signal power in neighboring symbols presented in [10], the Correlated Back-propagation (CBP) algorithm, in the context of a single channel transmission of 224 Gb/s PDM-16QAM over standard single-mode optical fiber (SMF). Linear equalization, DBP and CBP are compared in terms of OSNR penalty, maximum reached distance and computational complexity. II. OPTICAL SYSTEM MODEL The nonlinear Schrödinger equation (NLSE) describes the deterministic effects of optical fiber propagation of a single channel and single polarization signal [11]. For dual- polarization signals a pair of NLSE can be coupled in a so called Manakov system: , = 2 , + 2 , + 6 , (1) + , + , , where , is the electric field of a signal propagating in a given polarization, , is the electric field of a signal propagating in an orthogonal polarization, is the loss coefficient and describes fiber attenuation, and are related to CD and is the nonlinear parameter. Usually the CD effect is measured by dispersion parameter and dispersion slope , related to and by: = − 2 (2) = 2 + 4 (3) where is the carrier wavelength and is the speed of light. In many optical systems, especially those which employs Standard Single Mode Fibers (SSMF), the dispersion parameter is much higher than dispersion slope, and then slope impact is neglected [5]. Usually CD is the dominant linear effect in fiber propagation, and may cause severe inter-symbol interference. The last term of equation (1) describes intra- channel nonlinear effects, including non-coherent terms, responsible for Self Phase Modulation (SPM) and Intra- Channel Cross Phase Modulation (IXPM), and coherent terms responsible for Intra-Channel Four Wave Mixing (IFWM). SPM and IXPM are the dominant intra-channel nonlinear