Measuring Interactions Between Transport Protocols and Middleboxes Alberto Medina, Mark Allman, Sally Floyd ICSI Center for Internet Research {medina,mallman,floyd}@icir.org ABSTRACT In this paper we explore the current network environment with re- spect to how the network’s evolution ultimately impacts end-to-end protocols. The traditional end-to-end assumptions about the Inter- net are increasingly challenged by the introduction of intermediary network elements (middleboxes) that intentionally or unintention- ally prevent or alter the behavior of end-to-end communications. This paper provides measurement results showing the impact of the current network environment on a number of traditional and proposed protocol mechanisms (e.g., Path MTU Discovery, Ex- plicit Congestion Notification, etc.). We present results of mea- surements taken using an active measurement framework to study web servers. We analyze our results to gain further understanding of the differences between the behavior of the Internet in theory versus the behavior we observed through measurements. In ad- dition, these measurements can be used to guide the definition of more realistic Internet modeling scenarios. Categories and Subject Descriptors C.2.2 [Computer-Communication Networks]: Network Proto- cols; C.2.3 [Computer-Communication Networks]: Network Op- erations; C.2.5 [Computer-Communication Networks]: Local and Wide-Area Networks; C.2.6 [Computer-Communication Networks]: Internetworking General Terms Measurement, Design, Reliability, Standardization, Verification Keywords TCP, middleboxes, Internet, evolution 1. INTRODUCTION While the Internet’s architecture, protocols and applications are constantly evolving, there is often competing evolution between various network entities. This competing evolution can impact per- formance and robustness, and even halt communications in some Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. IMC’04, October 25–27, 2004, Taormina, Sicily, Italy. Copyright 2004 ACM 1-58113-821-0/04/0010 ...$5.00. cases. For instance, [23] shows that when setting up a TCP con- nection to a web server, attempting to negotiate the use of Explicit Congestion Notification (ECN) [27] interfered with connection es- tablishment for over 8% of the web servers tested in 2000. For such web servers, the client can only establish a TCP connection by re-attempting the connection without negotiating ECN usage. The connection failures in the presence of ECN negotiation were caused by firewalls configured to interpret the attempt to negotiate ECN as the signature of a port-scanning tool [10]. On the one hand, these firewalls can be seen as incorrectly associating new function- ality with one of the first appearances of that new functionality in an undesirable application. On the other hand, the firewalls can also be seen as doing their job of blocking unwanted traffic. This exam- ple shows the fundamental problem of different evolution paths that can cross to the detriment of smooth traffic flow on the Internet. In this paper, we investigate the evolution of TCP [26], the In- ternet’s most heavily used transport protocol, in the context of on- going changes to the Internet’s basic architecture. In particular, we study the ways in which so-called “middleboxes” (firewalls, NATs, proxies, etc.) — which change the Internet’s basic end-to-end prin- ciple [28] — impact TCP. We seek to elucidate unexpected interac- tions between layers and ways in which the Internet differs from its textbook description, including the difficulties various real-world “gotchas” impose on the evolution of TCP (and end-to-end proto- cols in general). The measurements presented in this paper also serve as lessons for efforts that wish to further evolve end-to-end protocols and the Internet architecture. In the study presented in this paper, we use active measurements to assess the capabilities supported by web servers (the primary data senders in web transactions) and their behavior in the context of the current Internet architecture on which they communicate. The remainder of this paper is organized as follows. Section 2 de- scribes related work on measurement studies of transport protocols. Section 3 describes the tools and methodology we use in our study. Section 4 explores interactions between middleboxes and transport protocols. Section 5 discusses additional results. Finally, Section 6 presents our conclusions, and discusses open questions and future work. 2. RELATED WORK This paper uses and extends the methodology from [23] on the TCP Behavior Inference Tool (TBIT). TBIT, the measurement tool used in our work, follows an earlier history of active probing of TCP. For instance, [8] treats TCP implementations as black boxes, observing how they react to external stimuli, and studying specific TCP implementations in order to assess the adherence to the spec- ification. There is also a considerable body of work on passive tests of