2886 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 27, NO. 14, JULY 15, 2009 Optimum Receiver Filters for Optical Fiber Systems With Polarization Mode Dispersion Ivan T. Lima, Jr., Member, IEEE, and Aurenice M. Oliveira, Member, IEEE Abstract—We use a semi-analytical receiver model combined with importance sampling applied to first-order polarization-mode dispersion (PMD) to show that the performance of optical fiber systems whose optical and electrical receiver filter bandwidths are optimized for polarization-mode dispersion in practical on-off-keyed systems is equivalent to the performance of the same systems in which the receiver filters are optimized for operation in the absence of PMD. We also observed an increase in the sensitivity of the performance with respect to variations in the receiver filter bandwidths in the return-to-zero format in systems with PMD when compared with PMD-free operation. Index Terms—Optical fiber communication, optical fiber polar- ization, polarization. I. INTRODUCTION P OLARIZATION-mode dispersion (PMD) is one of the effects that limit the data rate increase in a significant number of the currently deployed optical fiber communication systems from 10 to 40 Gbit/s. PMD compensators that consist of a fixed [1] and a variable [2] differential-group delay (DGD) ele- ment have been studied experimentally [3] and theoretically [4]. Most variable DGD elements consist of a beam splitter and one adjustable differential free-space propagation, but they have also been implemented in all-fiber based systems [5], [6]. Because of the stochastic nature of PMD, it is difficult to compensate for the performance degradation due to that impairment. Moreover, because of the wavelength dependence of PMD, several PMD compensators would be required in a wavelength-division mul- tiplexed system with a large number of channels. In the absence of cost-effective PMD compensators, it would be desirable to mitigate the performance degradation due to PMD by an ap- propriate choice of modulation format and receiver characteris- tics. In [7], it was shown through systems experiments using a 375 km transmission system with a first-order PMD emulator that the return-to-zero (RZ) modulation format is more robust to PMD-induced distortions than the non-return-to-zero (NRZ) format. In [8], the authors used a first-order PMD emulator to show how the penalty and the receiver sensitivity depend on the electrical receiver filter bandwidth and on the modulation Manuscript received June 22, 2008; revised December 11, 2008 and March 05, 2009. First published April 24, 2009; current version published July 09, 2009. I. T. Lima, Jr. is with the Department of Electrical and Computer Engi- neering, North Dakota State University, Fargo, ND 58108-6050 USA (e-mail: ivan.lima@ndsu.edu). A. M. Oliveira is with Electrical Engineering Technology, School of Tech- nology, Michigan Technological University, Houghton, MI 49931-1295 USA (e-mail: oliveira@mtu.edu). Digital Object Identifier 10.1109/JLT.2009.2018484 format for two DGD values. In [9], the authors showed how the outage probability due to first-order PMD depends on the elec- trical filter bandwidth in RZ and NRZ systems. In this paper, we show how the penalty depends on both the optical and the electrical receiver filters in NRZ and RZ op- tical fiber transmission systems using a semi-analytical receiver model [9], [10], in which we account for PMD-induced signal depolarization, combined with importance sampling applied to first-order PMD in which higher orders of PMD are also in- cluded [11]. We show that the combination of receiver filter bandwidths that minimizes the penalty margin at outage probability due to PMD in practical NRZ systems produces a performance that is very close to the one in which the receiver filters are optimized for PMD-free operation. We observed a negligible difference in the performance of a raised cosine RZ format with 50% duty cycle with these two sets of optimized filters. We observed a slightly larger, but still not significant, difference in the performance of an RZ system with 33% duty cycle in these two cases. We also observed an increase in the sensitivity of the performance with respect to variations in the receiver filter bandwidths in the RZ format in systems with PMD when compared with PMD-free operation. In [4] and [9], the outage probability was computed as- suming a large number of polarization hinges in the system. That assumption leads to a Maxwellian distribution of the DGD in a fixed wavelength over an extended period of time. Broroditsky et al. [12], showed that many practical systems have a relatively small number of polarization hinges, which leads to different distribution function for the DGD and, as a consequence, to different outage probabilities [13]. In this paper, we model systems with a large number of hinges to characterize the outage probability as a function of the receiver characteristics and of the modulation format. While the quan- titative results shown here do not apply to systems with small number of hinges, such as the one investigated in [14], the qualitative results should still hold for those systems. Since the tail of the distribution of both the DGD and of the second-order PMD decreases with the decrease of the number of hinges [11], the difference in the performance of optical fiber transmission systems with small number of hinges at outage probability due to PMD using receiver filters optimized in the presence of PMD and the same systems with the receiver filters optimized in the absence of PMD should be even smaller. Therefore, these results should also be applicable to optical fiber systems with small number of hinges. II. THEORETICAL MODEL AND SIMULATION PARAMETERS Computer simulations to investigate the impact of the optical and the electrical receiver filter bandwidths were carried out in 0733-8724/$25.00 © 2009 IEEE