A Cheating Detection Framework for Unreal Tournament III: a Machine Learning Approach Luca Galli, Daniele Loiacono, Luigi Cardamone, and Pier Luca Lanzi Abstract— Cheating reportedly affects most of the multi- player online games and might easily jeopardize the game experience by providing an unfair competitive advantage to one player over the others. Accordingly, several efforts have been made in the past years to find reliable and scalable approaches to solve this problem. Unfortunately, cheating behaviors are rather difficult to detect and existing approaches generally require human supervision. In this work we introduce a novel framework to automatically detect cheating behaviors in Unreal Tournament III by exploiting supervised learning techniques. Our framework consists of three main components: (i) an extended game-server responsible for collecting the game data; (ii) a processing backend in charge of preprocessing data and detecting the cheating behaviors; (iii) an analysis frontend. We validated our framework with an experimental analysis which involved three human players, three game maps and five different supervised learning techniques, i.e., decision trees, Naive Bayes, random forest, neural networks, support vector machines. The results show that all the supervised learning techniques are able to classify correctly almost 90% of the test examples. I. I NTRODUCTION In the context of on-line gaming, cheating refers to using artificial systems to gain a competitive advantage over the other players. Unluckily, cheating is a widespread phenomenon among on-line multiplayer games (especially among first person shooters) that affects negatively the game experience of honest players. Accordingly, several efforts have been made in the past years to find reliable and scalable solutions to this problem. Unfortunately, cheating behaviors are rather difficult to detect. Thus, the most successful and used approaches still heavily relies on the players’ collabora- tion as well as on the monitoring activity of the game-server administrators. In this scenario, machine learning is yet a poorly exploited technology and, so far, only few works [1], [2], [3], [4] focused on the application of machine learning techniques to this problem. In this work we introduce a methodology to automatically detect cheating behaviors in Unreal Tournament III, a slightly old but still popular first person shooter game. In partic- ular, our approach exploits supervised learning techniques to learn a cheating detection model from a labeled dataset. To this purpose, we first developed a rather sophisticated Luca Galli, Daniele Loiacono, Luigi Cardamone, and Pier Luca Lanzi are with the Politecnitco di Milano, Dipartimento di Elettron- ica e Informazione, Milano; email: luca.galli@mail.polimi.it, {loiacono,cardamone,lanzi}@elet.polimi.it, and customizable cheating system and, then, we designed a framework consisting of three main components: (i) an extended game-server, (ii) a processing backend, and (iii) an analysis frontend. The extended game-server is responsible for gathering the more relevant game data and sending them to the backend. The backend is in charge of preprocessing the collected data and applying the learned models to detect cheating behaviors. Finally, the frontend provides a user interface to perform both a real-time monitoring of the ongoing games and an off-line analysis of past games. Finally, we performed an experimental analysis to validate our framework. We collected a rather large dataset from several matches which involved three human players and three different game maps. Then, we applied five methods of supervised learning to build a model for the cheating detection and we assessed their performance on a test set. The reported results are very promising. All the five supervised learning techniques tested were able to classify correctly almost 90% of the cheating behaviors. In particular, the support vector machines and the Naive Bayes classifiers provided the highest accuracy with only one classification error out of 39 examples. The paper is organized as follows. In Section II we provide a brief overview of the related works in the literature. We introduce Unreal Tournament and the cheating behaviors respectively in Section III and in Section IV. In Section V we describe our framework while in Section VI we report the experimental results. Finally, we draw our conclusions in Section VII. II. RELATED WORK Several works in the literature (e.g., see [5], [6], [7], [8]) investigated protocols and system architectures to either detect or prevent cheating in on-line gaming. However, these works mainly focused on role-playing games (RPG) or real- time strategy games (RTS), while we focus on first person shooters. Concerning the application of machine learning techniques to detect cheating behaviors in first person shooters, only few works have been introduced so far in the literature. Yeung et al. [1] applied a dynamic Bayesian network to detect aimbot in Cube, an open source FPS game. In the Bayesian model designed by the authors, the aiming accuracy of the player only depends on the player’s movements, on the distance from the target, and on the presence of cheating behaviors. 978-1-4577-0011-8/11/$26.00 ©2011 IEEE 266