Scaling Ethernet speeds to 100 Gbits/s and beyond Arvinder S. Wander, 1, * Anujan Varma, 1 Drew Perkins, 2 and Vijay Vusirikala 2 1 Department of Computer Engineering, University of California, Santa Cruz, 1156 High Street, Santa Cruz, California 95064, USA 2 Infinera Corporation, 169 Java Drive, Sunnyvale, California 94089, USA * Corresponding author: awander@ucsc.edu Received October 28, 2008; revised December 19, 2008; accepted March 9, 2009; published April 7, 2009 Doc. ID 103346 Dramatic growth in Internet Protocol (IP) traffic demand is driving the need for new high-bandwidth IP interfaces. Today, router-to-router and router-to- transport system connections using Ethernet interfaces are limited to 10 Gbits/ s (10GE) or slower. Although techniques, such as link aggregation, allow a limited degree of extensibility beyond 10 Gbits/ s, they are limited in terms of scalability, introduce additional complexity, and reduce throughput efficiency. Discussion now centers on defining an Ethernet architecture that meets the needs of carriers and is conducive to implementation by the switch and server vendors. In this paper, we consider aggregation at the physical layer (APL) as a means to reuse existing 10GE physical layers (PHYs), while offering interface scalability to 100 Gbits/ s and beyond. With APL, aggrega- tion is performed at the PHY, whereby full Ethernet frames are transmitted across the aggregated PHYs in a parallel fashion. This ensures equal utiliza- tion of all links and allows aggregate bandwidth between nodes to scale with each new link added. We have demonstrated the applicability of such an ap- proach by implementing a 100 Gbits/ s interface using off-the-shelf compo- nents and running it over a live 4,000 km backbone network of a tier-1 Inter- net service provider. © 2009 Optical Society of America OCIS codes: 060.0060, 060.4250. 1. Introduction In the early 1990s, synchronous optical network/synchronous digital hierarchy (SONET/SDH) emerged as the optical standard for interoffice transport of time divi- sion multiplexing (TDM) voice circuits. Data-centric home and corporate local area networks (LANs), on the other hand, converged on Ethernet. Ethernet’s qualities of simplicity, scalability, and cost-effectiveness have made it ubiquitous in the LAN envi- ronment. These features of Ethernet, combined with recent advances in performance monitoring, manageability, and deterministic quality of service (QoS) support, are enabling Ethernet transport in carrier networks as well. Since the vast majority of the data traffic at the carriers is either Ethernet or Internet Protocol (IP), carriers are increasingly extending the Ethernet cloud from their customer LANs and into their wide area networks (WANs). Services like frame relay (FR), asynchronous transfer mode (ATM), and packet-over-SONET (PoS) used for data services in the past are being replaced by carrier Ethernet services and transport. In addition to the cost and simplicity advantages of Ethernet, it is compatible with multiprotocol label switching (MPLS), enabling a rich array of applications, such as virtual private LAN service (VPLS) for customers, as well as advanced traffic engineering options for carriers. In addition, recently initiated activities such as provider backbone bridges–traffic engi- neering (PBB-TE) enhance Ethernet with more carrier-class capabilities, including comprehensive performance monitoring, deterministic traffic routing, and fast and robust protection schemes [1]. Historically, Ethernet speeds have increased by a factor of 10 per generation—for example, 10 Mbits/s to 100 Mbits/s to 1 Gbit/s and to 10 Gbits/s. Today, 10 Gbits/s Ethernet (10GE) is available in many forms, depending on the application and the transmission distance—thousands of kilometers across a WAN, tens or hundreds of meters across a data center or central office (CO), a meter across a backplane, or cen- timeters between chips on a printed circuit board. In the WAN case, 10GE is generally carried as one serial data stream over optical fiber using native Ethernet or MPLS over either dark fiber or coarse wavelength division multiplexing (CWDM) or dense Vol. 8, No. 5 / May 2009 / JOURNAL OF OPTICAL NETWORKING 429 1536-5379/09/050429-9/$15.00 © 2009 Optical Society of America