J Supercomput (2007) 41: 17–40 DOI 10.1007/s11227-007-0101-0 Impact of protocol overheads on network throughput over high-speed interconnects: measurement, analysis, and improvement Hyun-Wook Jin · Chuck Yoo Published online: 24 February 2007 © Springer Science+Business Media, LLC 2007 Abstract Although extremely high-speed interconnects are available today, the tra- ditional protocol stacks such as TCP/IP and UDP/IP are not able to utilize the maxi- mum network bandwidth due to inherent overheads in the protocol stacks. Such over- heads are a big obstacle for high-performance computing applications to exploit high- speed interconnects in cluster environments. To address this issue, many researchers have been presenting analyses of protocol overheads and suggesting a number of optimization approaches to harness the TCP/IP suite over high-speed interconnects. However, to the best of our knowledge, there is no study that analyzes and optimizes the protocol overheads thoroughly in an integrated manner. In this paper, we exploit a set of protocol optimization mechanisms in an integrated manner while dealing with the full spectrum of the protocol layers from the transport layer to the physical layer. To evaluate the impact of each protocol overhead, we apply the optimization mechanisms one by one and perform detailed analyses at each step. The thorough overhead measurements and analyses reveal the dependencies between protocol over- heads. With our comprehensive optimizations, we show that UDP/IP can utilize more than 95% of the maximum network throughput a Myrinet-based experimental system can provide. Keywords UDP/IP · Myrinet · Clusters · High-speed interconnects · Programmable Network interface cards This work was supported by the Faculty Research Fund of Konkuk University in 2006. H.-W. Jin ( ) Department of Computer Science and Engineering, Konkuk University, 1 Hwayang-dong, Gwangjin-gu, Seoul 143-701, Korea e-mail: jinh@konkuk.ac.kr C. Yoo Department of Computer Science and Engineering, Korea University, 1, 5Ka, Anam-Dong, Sungbuk-Ku, Seoul 136-701, Korea e-mail: hxy@korea.ac.kr