J Comput Virol Hack Tech DOI 10.1007/s11416-017-0310-x ORIGINAL PAPER Detecting stealth DHCP starvation attack using machine learning approach Nikhil Tripathi 1 · Neminath Hubballi 1 Received: 16 June 2017 / Accepted: 18 October 2017 © Springer-Verlag France SAS 2017 Abstract Dynamic Host Configuration Protocol (DHCP) is used to automatically configure clients with IP address and other network configuration parameters. Due to absence of any in-built authentication, the protocol is vulnerable to a class of Denial-of-Service (DoS) attacks, popularly known as DHCP starvation attacks. However, known DHCP star- vation attacks are either ineffective in wireless networks or not stealthy in some of the network topologies. In this paper, we first propose a stealth DHCP starvation attack which is effective in both wired and wireless networks and can not be detected by known detection mechanisms. We test the effec- tiveness of proposed attack in both IPv4 and IPv6 networks and show that it can successfully prevent other clients from obtaining IP address, thereby, causing DoS scenario. In order to detect the proposed attack, we also propose a Machine Learning (ML) based anomaly detection framework. In par- ticular, we use some popular one-class classifiers for the detection purpose. We capture IPv4 and IPv6 traffic from a real network with thousands of devices and evaluate the detection capability of different machine learning algorithms. Our experiments show that the machine learning algorithms can detect the attack with high accuracy in both IPv4 and IPv6 networks. Keywords Anomaly detection · One-class classifiers · DHCP · DHCPv6 · DHCP starvation attack B Nikhil Tripathi phd1401101002@iiti.ac.in Neminath Hubballi neminath@iiti.ac.in 1 Discipline of Computer Science and Engineering, School of Engineering, Indian Institute of Technology Indore, Indore 453552, India 1 Introduction Dynamic Host Configuration Protocol (DHCP) [2] is used to obtain network configuration parameters including IP address from a DHCP server. This protocol is vulnerable to a class of Denial-of-Service (DoS) attacks popularly known as classical DHCP starvation attacks. Classical DHCP starva- tion attacks [4, 5] require a malicious client to inject a large number of IP requests using spoofed MAC addresses. For every such request received, a new IP address is released by a DHCP server. Thus, eventually DHCP server runs out of the IP addresses. However, it is not easy to launch classi- cal DHCP starvation attacks using spoofed MAC addresses in wireless networks as Access Point (AP) drops all the packets having source or destination MAC address pre- viously not associated with it. The only way to create a starvation attack is to precede and maintain association with AP for each spoofed MAC address. However, consider- ing the computational complexity involved in association and key exchange phase in WPA2 wireless networks, it is not feasible to perform multiple manual associations [11]. Moreover, various security features like port security [25] implemented on network switches can easily mitigate this attack by disabling the suspicious port on which multiple MAC addresses are seen at a time. On the other hand, Induced DHCP starvation attacks [6, 17], though effective in wire- less networks, can be mitigated by features like Dynamic ARP Inspection (DAI) [1] in wired networks as discussed in Sect. 3.3. In this paper, we propose a new stealth DHCP starva- tion attack that is effective in both IPv4 and IPv6 networks. This attack exploits IP address conflict detection scheme implemented on all DHCP clients. This attack is highly stealth as various popular security features in modern net- work switches can not detect the attack. Moreover, other 123