1 Dimitris Geneiatakis, Georgios Kambourakis, Tasos Dagiuklas, Costas Lambrinoudakis and Stefanos Gritzalis Laboratory of Information and Communication Systems Security Department of Information and Communication Systems Engineering University of the Aegean, Karlovassi, GR-83200 Samos, Greece Tel:+30-22730-82247 Fax: +30-22730-82009 Email:{dgen,gkamb,ntan,clam,sgritz}@aegean.gr Abstract— Internet telephony like any other Internet service suffers from security flaws caused by various implementation errors (e.g. in end-users terminals, protocols, operating systems, hardware, etc). These implementation problems usually lead VoIP subsystems (e.g. SIP servers) to various unstable operations whenever trying to process a message not conforming to the underlying standards. As Internet telephony becomes more and more popular, attackers will attempt to exhaustively “test” implementations’ robustness, transmitting various types of malformed messages to them. Since it is almost infeasible to avoid or predict every potential error caused during the developing process of these subsystems, it is necessary to specify an appropriate and robust, from the security point of view, framework that will facilitate the successful detection and handling of any kind of malformed messages aiming to destruct the provided service. In this paper, we adequately present malformed message attacks against SIP network servers and/or SIP end-user terminals and we propose a new detection “framework” of prototyped attacks’ signatures that can assist the detection procedure and provide effective defence against this category of attacks. I. INTRODUCTION It is well known, that both protocol implementations and network applications are not fully conformant with the underlying standards or that they contain development errors in their source code, which might “pollute” a network with incorrectly formed packets. A number of common TCP implementation problems are already documented in [1]. Thus, an attacker may employ malformed messages in order to cause “unstable operations” to the computing system. A malformed message is any kind of invalid or non-standard message, skillfully formed by the attacker in order to exploit and eventually take advantage of, any implementation gap or dysfunction might exist in the target system. Specifically for Internet applications or/and services, numerous distinct types of malformed message attacks have been already launched [2],[3]. Clearly, like any other Internet application or service, this problem cannot be avoided in Internet Telephony-Voice over IP (VoIP) implementations as well. Some research work that reveals security flaws caused by malformed messages in signaling protocols (such as H.323 and Session Initiation Protocol (SIP) implementations, have been already published in [4]-[6]. Moreover, attackers will keep trying to compromise the systems by utilizing properly adapted malformed messages. Malformed messages are characterized as a high-level type of attack that covers illegally formatted input. This security problem is often poorly understood and requires more research effort in order to be able to effectively protect implementations from this kind of attack. The PROTOS project [7] has made great strides to identify certain subclasses of malformed input. Processing malformed messages in VoIP networks can surprisingly give access to an unauthorized user or drive the provided service to various unstable operations and consequently cause Denial of Service (DoS). As a final point, the aforementioned issues implicitly affect the reliability and availability of VoIP service itself. This paper aims to describe malformed message attacks against SIP network servers or SIP end user terminal, proposing a framework, consisting of prototyped attacks’ signatures, that can assist to the identification and handling of such attacks. The rest of the paper is organized as follows: Section II describes special malformed messages that can be constructed in a SIP implementation. Section III briefly describes the procedure that an attacker can follow for launching a malformed message attack, while Section IV presents specific mechanisms for identifying and handling such attacks. Section V concludes the paper and provides pointers to future work. II. SIP MALFORMED MESSAGES SIP is an application-layer signalling protocol for creating, modifying, and terminating multimedia sessions between one or more participants [8]. A SIP message can be either a request or an acknowledgment to a corresponding request, A Framework for Detecting Malformed Messages in SIP Networks