WDMJTDM Optical Packet Switched Network for Supercomputing Q. Yang, M. F. Arend, G. D. Hughes*, F. G. Johnson** and K. Bergman Electrical Engineering Department, Princeton University, Princeton, NJ 08544, *Depament of Defense, Fort Meade, MID 20755, **Laboratory for Physical Sciences, University of Maryland, College Park, MD 20740 ABSTRACT A new optical packet switching network and its enabling technologies are investigated for implementation in a Petaflops scale supercomputer system. We capitalize on the immense bandwidth of the optical fiber interconnects by deploying WDM/TDM packet payloads. To accommodate current optical switching technologies, the routing operations in the network are drastically simplified and the need for buffering is completely eliminated. This paper presents the experimental demonstration of the routing within the unique packet switched architecture. Multiple node hops are demonstrated in a node test-bed environment with a re-circulating loop configuration. Keywords: optical packet switch, WDM routing 1. INTRODUCTION The current state-of-the-art supercomputing systems operating at a Teraflops scale are already becoming insufficient for many important applications that require far greater capability, on the scale of a Petaflops. Teraflops scale massively parallel processors (MPPs) are being implemented in CMOS technology at a cost of approximately 100 million dollars and power consumption of megawatts. Direct scaling of these technologies to the Petaflops regime would be prohibitive in cost, power consumption, and size. To address this, a major research project is being conducted to develop and evaluate a hybrid technology multithreaded (HTMT) architecture incorporating advanced devices and employing proactive runtime adaptive latency management mechanisms j1j[2)• Within the HTMT architecture we are designing and implementing an optical interconnections network that forms the communications infrastructure among the thousands of high-speed processor in memory (PIM) elements. In the following we provide a brief overview of the HTMT computing system and a description of the optical network experimental test-bed. The HTMT hybrid computing structure synthesizes a balanced mix of complementing technologies that offer the potential to resolve some of the key barriers (cost, power, throughput bandwidth, latency) to Petaflops computing. An overview of the HTMT machine architecture is shown in Figure 1 . The computing processors employ Superconductor Rapid Single Flux Quantum (RSFQ) logic that can operate in the region of 100GHz with low power consumption (31• At the next level in the memory hierarchy are processor-in-memory (PIM) SRAMs and DRAMs permitting fully integrated memory and logic cells on a single chip, exposing the full memory bandwidth of the internal row buffers and significantly reducing packaging and pin counts [4j• A new packet switching optical network and its enabling technologies is the subsystem focused on in this paper. This network provides interconnection among tens of thousands of ports with an operating throughput bandwidth exceeding a Petabyte/sec and latencies less than 100 nanoseconds [5] At the top of the HTMT memory hierarchy are holographic photorefractive systems that may be able to store up to 100 Gbits in a cubic centimeter and deliver bandwidths approaching a terabit per second 6j• Advanced architecture structures and mechanisms are required to achieve high processor efficiency as the speed of processor technology increases relative to that of high density memory technology. The challenge of latency management is aggravated by the exploitation of exotic technologies for which this ratio may approach a million. The HTMT architecture addresses this problem through an innovative multi-level multithreaded execution model 71• In Optics in Computing 2000, Roger A. Lessard, Tigran Galstian, Editors, SPIE Vol. 4089 (2000) 0277-786X/0O/$15.0O 555