Alec Yasinsac, Justin Childs Florida State University Computer Science Dept. yasinsac@cs.fsu.edu, childs@cs.fsu.edu Abstract In this paper, we show how a novel tool for analyzing classical cryptographic protocols can be used to model and analyze the more complex Internet security protocol families. We discuss the modifications that were necessary in the tool and how the tool illuminates flaws in the Transport Layer Security (TLS) protocol. We also show how a flaw carried over to TLS from the Secure Sockets Layer protocol (the predecessor of TLS) was illustrated using our tool. 1. Introduction The seminal paper by Needham and Schroeder [11] triggered the field of cryptographic protocol analysis in 1978. For twenty years, that analysis focused on simple, serial protocols in a structured and well-understood environment (e.g. [2]). This analysis has been very successful in identifying flaws in well-known and widely used protocols. In addition, much was learned about the nature of cryptographic protocols and characteristics that reflect strengths and weaknesses [1], [13]. Automatic analysis tools, such as the NRL Protocol Analyzer, developed by Meadows [9] and the AAPA2 developed by Brackin [3], have extended these ideas to software tools that can aid in the analysis of cryptographic protocols. In the past five years, these simple protocols have given way to more complex protocol suites, suitable for E- Commerce applications and general use on the Internet. These protocols have characteristics that make use of early analysis methods difficult or impossible. For example, early analysis of cryptographic protocols by formal methods is limited to protocols where the actions of individual parties cannot be changed by actions within the protocol. Many security protocols in use or proposed for widespread use, such as Secure Electronic Transaction (SET), Internet Key Exchange (IKE), Secure Sockets Layer (SSL), and TLS, offer optional sub-protocols that are agreed upon by protocol participants during protocol execution. Interactions between the sub-protocols may allow the protocol to be subverted. If the principals in the protocol can be confused as to what protocol is being run, an intruder can increase the potential number of interactions between the principals. These new interactions may reveal an unforeseen weakness in the protocol. Wagner and Schneier [14] discovered a weakness of this type in the Secure Socket Layer (SSL) protocol. In order to fully analyze these protocols, interactions between the sub-protocols must be considered. Unfortunately, analyzing sub-protocol interactions can lead to rapid increases in the cost of analysis as the number of possible interactions grows rapidly with the addition of sub-protocols. This makes the analysis difficult and costly. To date, Meadows [9] appears to have had the only success in performing this type of analysis with her work on IKE and SET using the NRL protocol analyzer. The scarcity of tools examining sub-protocol interactions can be explained in part by the origins of the tools used to analyze security protocols. Until recently, security protocols have only contained sequential control. Hence, analysis tools had no reason to create ways to model branching operations. The Cryptographic Protocol Analysis Language Evaluation System (CPAL-ES) is based on a formal programming method, so a branch operator was included from the outset. This inclusion allowed the tool to represent and analyze protocols that include sub-protocols. If not for the problem of state explosion, which is an obstacle to extending all tools, the CPAL-ES system was ready to analyze branching protocols before the extension in this work were made. Despite these obstacles, from work done by Wagner and Schneier that found a vulnerability caused by the interaction of two sub-protocols [14]. Another paper by Kelsey, Schneier, and Wagner proved the existence of an insecure protocol match for any other protocol [8]. It is apparent that tools that can analyze sub-protocol interactions have a target-rich environment in which to discover flaws in security protocols. Analyzing Internet Security Protocols