An Experiment in Automatic Game Design Julian Togelius and J¨ urgen Schmidhuber Abstract—This paper presents a first attempt at evolving the rules for a game. In contrast to almost every other paper that applies computational intelligence techniques to games, we are not generating behaviours, strategies or environments for any particular game; we are starting without a game and generating the game itself. We explain the rationale for doing this and survey the theories of entertainment and curiosity that underly our fitness function, and present the details of a simple proof- of-concept experiment. I. I NTRODUCTION Can computational intelligence (CI) help designing games? One is tempted to answer “Yes, obviously, the whole field of Computational Intelligence in Games (CIG) is devoted to this, isn’t it?” However, the majority of CIG research is concerned with learning to play particular games as well as possible. There is nothing wrong with this type of research; indeed, it is very valuable for the science of artificial intelligence. Games provide the type of deep but accessible reinforcement learning problems that we badly need in order to develop better CI algorithms and controller representations. But these types of studies are less interesting to game development in general, and to commercial game developers in particular. This is because commercial game developers are, in gen- eral, not interested in ways of making NPCs (non-playing characters, e.g. opponents) play better. This is because it is usually possible to make them play well enough using just a tiny bit of (virtually undetectable) cheating, such as giving the NPC more information than the human player, or slightly superior abilities. Further, having NPCs that play as well as possible is no end in itself; if the opponents are too hard to beat, the game quickly becomes uninteresting for human players. Of course, there are exceptions. In particular, many strat- egy games (such as Civilization or Starcraft) are so com- plicated that it’s hard to come up with NPC AI that plays competitively against good human players, without resorting to most blatant cheating. In such cases, CI techniques for beating the game better might contribute to the quality of the game. And let us not forget that many games are representations of real-life problems (e.g. battle tactics or vehicle control), and advances in learning how to play games might be transferable to related real-life problems. But for most game genres, such as racing games, platformers, puzzle games or first person shooters, there does not seem to be any interest from game developers in learning to play the game better per se. Both authors are with IDSIA, Galleria 2, 6928 Manno-Lugano, Switzer- land. J¨ urgen Schmidhuber is also with TU Munich, Boltzmannstr. 3, 85748 Garching, M¨ unchen, Germany. emails: {julian, juergen}@idsia.ch Now, there is certainly other research being carried out in the CIG field that is more directly relevant to real game development (and often dependent on research done in learning to play games, which thus becomes indirectly relevant to game development). For example, we have CI techniques proposed to generate NPC controllers that play interestingly as opposed to just well [1], [2]; CI techniques for automatically finding exploits/bugs in games [3]; CI techniques for modelling the behaviour of human players[4], [5]; CI techniques for making NPCs trainable by human players [6]; and techniques for generating the content of a game, such as tracks, levels or mazes [4], [7]. While the above techniques all represent relevant research directions for game design, they all assume that there is a game there to begin with. Before we let CI loose on generating behaviours, strategies or environments, or testing the game, it needs to have something to play with. However, game design is concerned with all aspects of the game. In particular, it is designed with the very heart of every game, that which defines the game: its rules [8]. Game rules determine when the game begins and when it ends. They determine what the player can and can’t do, and (together with the actions of other players) what happens as a result of the player’s actions. The complexity of rules vary from something as simple as checkers, which can be expressed in a paragraph of text, to something as complex Starcraft, Counter-strike or other modern video games, where the rules cannot be expressed in its entirety without describing the whole game engine. Of course, there are borderline cases regarding what is and what isn’t a rule. While the set of legal capture moves in Checkers are clearly rules, would the throwing speed and blast radius of a grenade on Counter-strike count as rules, aspects of the environment or something else? In this paper, we will take a fairly liberal and inclusive view of what constitutes rules for a game in our argument, whereas in the experiment we will err on the side of exclusiveness and adopt a stricter view of what constitutes a rule. In this paper, we describe an initial proof-of-concept experiment where we evolve the rules for a game. To our best knowledge, this represents the first time rules have been evoled for a single-player game; the first time rules have been evolved for a non-board game; and the first time a fitness function based on learning progress has been used for evolving game rules. An interesting recent study describes the evolution of board game rules using a fitness function based on measurements of historical play patterns [9]. It is worth noting that both the type of game and the fitness function differ drastically between these two independently conceived and executed studies. 978-1-4244-2974-5/08/$25.00 ©2008 IEEE 111