Design of a System for Real-Time Worm Detection Bharath Madhusudan, John Lockwood Department of Computer Science and Engineering Washington University, St. Louis {bharath, lockwood} @arl.wustl.edu Abstract Recent well publicized attacks have made it clear that worms constitute a threat to Internet security. Systems that secure networks against malicious code are expected to be a part of critical Internet infrastructure in the future. Intrusion Detection and Prevention Systems (IDPS) currently have limited use because they can filter only known worms. In this paper, we present the design and implementation of a system that automatically detects new worms in real-time by monitoring traffic on a network. The system uses Field Programmable Gate Arrays (FPGAs) to scan packets for patterns of similar content. Given that a new worm hits the network and the rate of infection is high, the system is automatically able to detect an outbreak. Frequently occuring strings in packet payloads are instantly reported as likely worm signatures. I. Introduction Internet worms work by exploiting vulnerabilities in operating systems and application software that run on end systems. The attacks compromise security and degrade network performance. They cause large economic losses for businesses resulting from system down-time and loss of worker productivity. Several approaches to defend against worms are described in [1]: • Prevention : Worms usually work by exploiting vul- nerabilities in software. Prevention involves writing secure code. Much work has been done by the Language Security community towards achieving this goal. • Treatment : Programmers analyze the vulnerability that the worm exploits and release a “patch” that fixes it. However, it takes time to analyze and patch software. In addition, many end users may never apply the patch. As a result, a significant fraction of machines in the network remain vulnerable. • Host-based containment systems : An example of host-based defense would be one that restricts the number of active connections. This would restrict the number of worm “probes” that hit other vulnerable machines on the Internet. A problem with host-based defense is that it is not easy to deploy. It requires that a privileged process be run and maintained on every end system in the network. • Containment by the network : The advantage of using network based containment is that a small number of IDPSs can protect an entire corporate or university de- partment network. Current intrusion detection systems use signatures to detect only known worms. Hence the network is left vulnerable to unknown worms. This paper focusses on the design and implementation of a containment system that protects the network against unknown worms. Design goals of the system include: • Low reaction time : The system should detect anoma- lies quickly and begin blocking malicious data flows so that damage caused by the worm can be limited. • High throughput : Systems must keep up with today’s high speed links, like Gigabit Ethernet and OC-48, in order to monitor all traffic in real-time. Software- based systems can perform only a limited number of operations within the time period of a packet transmission. This necessitates the use of hardware. • Low cost : The system should fit within the limits afforded by today’s FPGAs and ASICs. • Low false positive rate : The system must generate a minimum number of false positives. Otherwise, warnings will be ignored and/or legitimate traffic will be blocked. • Robust to simple countermeasures : The system should have the ability to detect polymorphic worms that work by inserting no-ops and random code in the worm payload. Our system is designed to work in tandem with an IDPS such as the one presented in [2]. The short time taken to detect a worm signature allows an IDPS to be