2310 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 29, NO. 15, AUGUST 1, 2011 Transmission of 9 138 Gb/s Prefiltered PM-8QAM Signals Over 4000 km of Pure Silica-Core Fiber Roberto Cigliutti, Member, IEEE, Enrico Torrengo, Gabriella Bosco, Member, IEEE, Nunzio Paolo Caponio, Andrea Carena, Member, IEEE, Vittorio Curri, Member, IEEE, Pierluigi Poggiolini, Member, IEEE, Yoshinori Yamamoto, Takashi Sasaki, and Fabrizio Forghieri, Member, IEEE Abstract—We experimentally demonstrated the application in ultrahigh spectral density long-haul optical systems of tight prefiltering to generate “quasi-Nyquist wavelength division multiplexing” signals together with the use of pure-silica core fiber and hybrid Raman/erbium-doped fiber amplification. The transmission of polarization-multiplexed quadrature-amplitude modulation with 8 symbols (PM-8QAM) at 138 Gb/s and channel spacing equal to 1.22 (symbol rate) was demonstrated, reaching over 4000 km at net spectral efficiency (SE) of 4.1 b/s/Hz. In our experiment, we achieved to date highest per-carrier symbol rate (23 Gbaud), SE (4.1 b/s/Hz), and (SE) (distance) product (16655 b/s/Hz km) for non-orthogonal frequency-division multiplexing transmission using PM-8QAM modulation. Index Terms—Coherent detection, Nyquist wavelength division multiplexing (Nyquist-WDM), polarization multiplexing, polariza- tion-multiplexed quadrature-amplitude modulation with 8 sym- bols (PM-8QAM), pure silica-core fiber (PSCF). I. INTRODUCTION R ECENTLY, in order to meet the ever growing demand for data transmission capacity over optical channels, there has been an increasing interest in the investigation of terabit superchannels [1]–[5]. According to this solution, a number of optical channels (subcarriers) are seamlessly aggregated to form “superchannels,” which would then be routed optically through the network as a single entity. For such a technique to be effec- tive in increasing capacity, terabit superchannels must be spec- trally compact: subcarriers must be tightly packed with spacing close or equal to the Nyquist limit. This result can be, for instance, achieved using one of the different flavors of coherent orthogonal frequency domain mul- Manuscript received March 04, 2011; revised May 21, 2011; accepted May 26, 2011. Date of publication June 09, 2011; date of current version July 22, 2011. This work was supported by CISCO Systems (SRA contract) and by EU- ROFOS, a Network of Excellence funded by the European Commission through the seventh ICT-Framework Programme. R. Cigliutti, G. Bosco, N. P. Caponio, A. Carena, V. Curri, and P. Poggi- olini are with the Dipartimento di Elettronica, Politecnico di Torino, Torino 10129, Italy (e-mail: roberto.cigliutti@polito.it; gabriella.bosco@polito.it; paolo.caponio@tiscali.it; andrea.carena@polito.it; vittorio.curri@polito.it; pierluigi.poggiolini@polito.it). E. Torrengo was with Politecnico di Torino, Torino 10129, Italy. He is now with R&D Department, Nokia Siemens Networks, Amadora 2720-093, Portugal (e-mail: enrico.torrengo@polito.it). Y. Yamamoto and T. Sasaki are with Sumitomo Electric Industries, Yokohama 244-8588, Japan (e-mail: yamamoto-yoshinori@sei.co.jp; sasaki-takashi@sei.co.jp). F. Forghieri is with Cisco Photonics Italy s.r.l., Monza 20900, Italy (e-mail: fforghie@cisco.com). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/JLT.2011.2159193 tiplexing (CO-OFDM) (with [6] or without [3] a reduced guard interval), i.e., channels with sharp transitions in time-domain packed at symbol rate spacing in the frequency domain. In fact, in conventional OFDM the frequency spacing equal to is a required condition in order to obtain orthogonality among subcarriers [7], [8]. Alternatively, a high spectral efficiency (SE) can also be achieved using a standard wavelength division multiplexing (WDM) approach with very tight channel spacing. In the latter case, the orthogonality between side channels is achieved thanks to narrow spectral shaping, which can be efficiently per- formed either in the optical domain, through narrow transmitter (Tx) optical filtering, or in the electrical domain, combining digital signal processing (DSP) and digital-to-analog conver- sion. This technique, named Nyquist-WDM, has been widely used in radio links for decades and has been lately proposed and demonstrated for optical links too [4], [5], [9]–[13]. In order to relax the constraints on the steep spectral shaping requested to satisfy the orthogonality condition and to minimize crosstalk between the subcarriers, the channel spacing can be larger than , accepting little loss in spectral efficiency [14]. Successful experiments exploiting Nyquist-WDM (with channel spacing equal to the symbol rate) or “quasi-Nyquist- WDM” (with channel spacing slightly larger than the symbol rate), based on polarization-multiplexed quadrature-phase shift keying (PM-QPSK) modulation format, have recently been announced [9], [10], reaching trans-pacific distances. In this paper, we extend the experimental investigation of quasi-Nyquist-WDM long-haul transmission to polariza- tion-multiplexed quadrature-amplitude modulation with 8 symbols (PM-8QAM). In particular, we analyze the transmis- sion of nine PM-8QAM channels at 23 Gbaud (corresponding to a raw bit rate of 138 Gb/s per channel and a total raw capacity of 1.24 Tb/s) with spacing GHz, i.e., , where is the symbol rate. An original scheme was used for the 8QAM modulator, whose structure is described in Section II-B. The experiments were performed using a recirculating loop setup based on pure silica-core fiber (PSCF) [15] and hybrid Raman/erbium-doped fiber amplification. We tested two types of PSCF, both provided by Sumitomo: the commercial Z-PLUS fiber and an innovative large-effective-area PCSF prototype (LA-PSCF fiber). We assumed to use a hard-decision forward error correction (FEC) with 20% overhead and an in-service bit error rate (BER) threshold equal to . Such a hypothesis was justi- fied by state-of-the-art advanced hard-FECs technology, based on continuously interleaved BCH codes [16], [17], which is 0733-8724/$26.00 © 2011 IEEE