Proceedings of the 2011 Winter Simulation Conference S. Jain, R. R. Creasey, J. Himmelspach, K. P. White, and M. Fu, eds. MAY THE BEST MAN WIN: SIMULATION OPTIMIZATION FOR MATCH-MAKING IN E-SPORTS Ilya O. Ryzhov Robert H. Smith School of Business University of Maryland College Park, MD 20742, USA Awais Tariq Warren B. Powell Operations Research and Financial Engineering Princeton University Princeton, NJ 08544, USA ABSTRACT We consider the problem of automated match-making in a competitive online gaming service. Large numbers of players log on to the service and indicate their availability. The system must then find an opponent for each player, with the objective of creating competitive, challenging games that do not heavily favour either side, for as many players as possible. Existing mathematical models for this problem assume that each player has a skill level that is unknown to the game master. As more games are played, the game master’s belief about player skills evolves according to a Bayesian learning model, allowing the game master to adaptively improve the quality of future games as information is being collected. We propose a new decision-making policy in this setting, based on the knowledge gradient concept from the literature on optimal learning. We conduct simulations to demonstrate the potential of this policy. 1 INTRODUCTION With the appearance of affordable broadband Internet, competitive online gaming or e-sports has become a massive cultural phenomenon, as well as a profitable business venture. A study by Huhh (2008) documents the growth in revenues of South Korean online game company NCSoft, from $559 million in 2000 to over $2 billion in 2004. In 2005, Microsoft’s Xbox Live online service had over 2 million subscribers (Herbrich et al. 2006). The competitive strategy game Starcraft II, released in 2010, has over 2.5 million players listed in its ranking system (SC2 Rankings 2011) as of this writing. Competition is at the heart of e-sports. Online game services frequently create and display rankings of players (as seen above for the case of Starcraft II). Outside organizations create their own rankings (KeSPA 2011), used to evaluate professional players. Ranking has a great impact on the experience of even casual players. Large online systems automate the process of match-making: a player logs on to the system and submits a request to play a game, whereupon the system finds an opponent with no further input from the player. The goal is to create fair and challenging games by matching players of similar skill level. However, a player’s skill level is not known exactly to the system, and must be inferred from the player’s match history. Methods for statistical modeling of player skills and prediction of game outcomes date back to the work by Elo (1978), which focuses on rating chess players. The emergence of e-sports has sparked a new interest in such methods. Studies by Herbrich et al. (2006) and Dangauthier et al. (2007) adopt a Bayesian perspective for modeling player skills. The game master has a Bayesian belief about the skill of each player, and adjusts this belief as new games are played. Furthermore, the Bayesian beliefs can be used to estimate the quality of a particular match-up, enabling the game master to make match-making decisions. The resulting TrueSkill TM ranking system has been implemented by Xbox Live. The match-making problem can be interpreted as a variation on ranking and selection, a fundamental problem in simulation optimization (see e.g., Swisher et al. 2000 or Hong and Nelson 2009 for an introductory overview, or Kim and Nelson 2007 for a view geared more toward recent advances). In