The Moonraker Study: An Experimental Evaluation of Host-Based Deception Temmie B. Shade, Andrew V. Rogers Kimberly J. Ferguson-Walter Laboratory for Advanced Cybersecurity Research tbshade@tycho.ncsc.mil Sara Beth Elson, Daniel K. Fayette Kristin E. Heckman The MITRE Corporation selson@mitre.org Abstract Cyber deception has been discussed as providing enhanced cyber defense. This human subjects research, one of the first rigorously controlled studies on this topic, found that host-based deception was effective at preventing completion of a specific exfiltration task against a virtual network. In addition to impeding progress and preventing success, the deception resulted in increased confusion and surprise in the participants. This study provided the necessary rigor to scientifically attest to the effectiveness of cyber deception for cyber defense with computer specialists. 1. Introduction Traditional network defense practices are proving to be increasingly ineffective at stopping the relentless and innovative offensive maneuvers of cyber attackers. Cyber deception is a growing part of the defender’s arsenal aiming to slow down or prevent compromise by introducing confusion, frustration, or other psychological effects to the cyber attackers themselves. Most research on cyber deception has been on honeypots [1] or honeynets [2], decoy documents [3], or decoy network nodes [4]. However, there are many additional avenues defenders may take to muddle opponents and grant their systems an air of uncertainty. In our study, we devised a novel approach of deceiving network intruders and measuring the effects on their campaign in terms of both success and cognition in an experiment. We have utilized a tool called Moonraker to intercept specific commands and react in deceptive ways. The experiment was conducted as a technical class teaching red team methodologies. Unbeknownst to the participants, Moonraker was utilized in the exercise portion of the class for half of the participants, configured to intercept the primary commands needed to complete the DISTRIBUTION A. Approved by AFRL for public release: distribution unlimited. Case Number 88ABW-2019-2863. task. Completion required participants to enumerate a network’s hosts, connect to one of the hosts, copy a malicious file to the host, execute the file, and retrieve its output. A post-exercise survey was given to capture participants’ feelings and feedback on several fronts, including: emotional experiences (doubt, confusion, and frustration), past expertise in relevant technical skills, questions about their suspicion of the use of deception in the exercise, and, to further reinforce the cover story of the experiment being a training class, questions about the instructive material and exercise. We hypothesized that the added deceptive actions would impede attacker progress and create a more frustrating and time-consuming attacker experience. While other researchers have made similar claims [5] [6], this is one of the first rigorously controlled experiments to examine the effectiveness of deception for cyber defense. To test our hypothesis, there were several metrics that were computed, including success on task, command success ratios, and self-reported emotions from the survey. It is generally not feasible to collect some of these metrics when an adversary is operating on a network, nor is it possible to control the environment as needed to attribute the cause to the experimental manipulation. As such, this research study provides a valuable contribution–scientific validation of the efficacy of cyber deception for defense. 2. Related Work A variety of cyber deception techniques have been developed to thwart attackers, such as honeypots [1] and decoys [4]. Over the past several years, researchers have sought to determine the effectiveness of deceptive defenses by conducting studies with human participants. These studies have primarily focused on determining the realism of deception, measuring the difference in time on deceptive versus real assets, and assessing the abilities of deceptive techniques to detect attackers. Sample sizes were often small and most did not employ control conditions for comparison [6], thus they lacked Proceedings of the 53rd Hawaii International Conference on System Sciences | 2020 Page 1875 URI: https://hdl.handle.net/10125/63970 978-0-9981331-3-3 (CC BY-NC-ND 4.0)