HInjector: Injecting Hypercall Attacks for Evaluating VMI-based Intrusion Detection Systems Aleksandar Milenkoski 1 , Bryan D. Payne 2 , Nuno Antunes 3 , Marco Vieira 3 , and Samuel Kounev 1 1 Karlsruhe Institute of Technology, Germany (e-mail: milenkoski, kounev@kit.edu) 2 Nebula Inc., USA (e-mail: bdpayne@acm.org) 3 University of Coimbra, Portugal (e-mail: nmsa, mvieira@dei.uc.pt) 1. INTRODUCTION Virtual machine introspection (VMI) is a mechanism for monitoring the states of guest virtual machines (VMs) from a virtualization host. Its use for intrusion detection is an emerging trend that brings many benefits, such as the pos- sibility to monitor guest VMs in a transparent manner for attackers. However, a major issue is the evaluation of the attack detection accuracy of VMI-based intrusion detection systems (IDSes), e.g., [1], for detecting virtualization-related attacks, such as attacks targeting virtual machine monitors (VMMs). VMMs’ attack surfaces are very narrow making them difficult to penetrate. As a result there are only a few publicly available exploits that demonstrate attacks against VMMs. We believe that the automated artificial injection of malicious system activities with respect to representative attack models is a promising approach towards overcoming this issue. A malicious VM user may execute an attack against the underlying VMM via several attack vectors such as device drivers, VM exit events, or hypercalls. Hypercalls are soft- ware traps from a kernel of a paravirtualized guest VM to the VMM. They enable the intrusion in VMMs, initiated from a malicious guest VM kernel, in a procedural man- ner through VMMs’ well-defined hypercall interfaces. This makes the automated injection of hypercall attacks feasible. Further, given the potential high severity of hypercall at- tacks, the evaluation of the effectiveness of IDSes for detect- ing and preventing such attacks is crucial. The exploitation of a vulnerability in a VMM’s hypercall handler typically leads to altering the VMM’s memory enabling, for example, the execution of malicious code with VMM privileges. In this paper, we present HInjector, a customizable frame- work for injecting hypercall attacks during regular operation of a paravirtualized guest VM in a Xen-based environment. Besides the fact that it is a popular VMM, we chose Xen as a VMM targeted by injected hypercall attacks because most of the documented vulnerabilities of VMMs’ hypercall handlers are those of Xen. This enables the construction of representative attack models. Further, Xen is open-source software and thus its codebase can be modified, which, as shown later, is necessary for injecting hypercall attacks. The attacks injected by HInjector conform to attack mod- els based on existing Xen vulnerabilities (e.g., CVE-2008- 3687, CVE-2012-5513). The goal of HInjector is to exercise the sensors of a typical VMI-based IDS. An IDS may use VMI to monitor guest VM’s virtual CPU (vCPU) registers in order to identify, for example, anomalous hypercall pa- rameter values. It may also monitor guest VM’s memory and inspect, for example, call stacks to identify irregular hypercall call sites. Since VMI-based IDSes do not monitor states of VMMs, HInjector does not inject the effects that hypercall attacks may have on the state of a given attacked VMM (e.g., altered VMM’s memory). To the best of our knowledge, we are the first to consider the injection of (hypercall) attacks targeting VMMs. Pham et al. [4] and Le et al. [2] focus on injecting generic soft- ware faults directly into VMMs. The latter is not suitable for evaluating VMI-based IDSes since, as mentioned above, such IDSes do not monitor states of VMMs. Also, generic software faults are not representative of faults caused by ex- ploiting security vulnerabilities. Oyama et al. [3] present a framework for injecting effects of attacks targeting guest VMs, however, they do not take into account hypercall at- tacks and attacks targeting VMMs in general. 2. HINJECTOR DESIGN We constructed attack models based on analyzing previously discovered vulnerabilities of Xen’s hypercall handlers. We analyzed relevant publicly disclosed vulnerability reports, the earliest dating back to 2008, identifying patterns of VM activities for executing hypercall attacks. We then catego- rized the identified patterns into attack models. To con- struct attack models, we also took into consideration hy- percall characteristics relevant for intrusion detection. We distinguish the following attack models: i) Invoking hypercalls from irregular call sites. VMI- based IDSes (e.g., [1]) may consider hypercalls invoked from call sites unknown to them, e.g., an attacker’s loadable ker- nel module (LKM), as malicious. We model the typical sce- nario where an attacker loads a LKM to execute attacks from kernel mode, which includes hypercall attacks. ii) Invoking hypercalls with anomalous parameter values a) outside the valid value domains, or b) crafted for ex- ploiting specific vulnerabilities (not necessarily outside the valid value domains). This attack model is based on the Xen vulnerabilities described in CVE-2008-3687, CVE-2012- 3516, CVE-2012-5513, and CVE-2012-6035. iii) Invoking a series of hypercalls in irregular order, in- cluding repetitive execution of a single or multiple hyper- calls. This attack model is based on the Xen vulnerability described in CVE-2013-1920. The repetitive execution of hypercalls, for example, requesting system resources, is an easily feasible attack which may lead to resource exhaustion