Visual Procedural Content Generation with an Artificial Abstract Artist Matthew Guzdial, Duri Long, Christopher Cassion, Abhishek Das College of Computing Georgia Institute of Technology Atlanta, GA 30308 USA {mguzdial3, duri, ccassion3, abhshkdz}@gatech.edu Abstract We present Pollite (Pollock-lite), an artificial abstract artist with the capability to evaluate, augment, and gen- erate video game visual elements. Our system is based on a cognitive model of abstract artists built from self- reports and interviews. The main intelligence behind our system is a Convolutional Neural Net (CNN), a deep neural network approach that has shown great success in image tagging tasks and that can learn associations between shapes, colors, and concepts. We demonstrate initial results with our system across three case studies. Introduction Procedural content generation (PCG) refers to the body of techniques for algorithmically generating video game con- tent through either designer-defined rules and heuristics or based on machine learned models of existing content (Hen- drikx et al. 2013; Summerville et al. 2017). The majority of PCG systems, with some notable exceptions (Cook, Colton, and Gow 2016a), do not attempt to replicate the internal pro- cesses of human creative designers, but instead focus on a particular game development task (e.g. video game level creation). In addition, the majority of PCG work focuses on function over form in large part due to functional ele- ments affording easier evaluation (Cook and Smith 2015). By function vs. form we indicate the divide between focus- ing on structural, quantifiable elements in games like level generation focused on playability over altering the visual ap- pearance of a game. A PCG system capable of generating, altering, and eval- uating the visual aesthetics of a video game could be an invaluable tool for video game developers, particularly for those developers who are not themselves skilled in visual design or do not have access to quality visual design knowl- edge. Such a PCG system would benefit in being modeled after a human creative visual designer, in order to facilitate better collaboration with human developers. Thus a human developer could communicate their aesthetic intentions to the automated visual designer in natural language. In this paper we present an artificial abstract artist, Pollite (Pollock-lite). Pollite is capable of generating, altering, and evaluating visual components of video games, and is based on a cognitive model of a human abstract artist. We focus on abstract art for our cognitive model in order to avoid tying our system to a particular style of game art. We learn the visual aesthetic knowledge for our system from real-world images (photographs) with a convolutional neural network (CNN), a deep learning approach. The rest of this paper is organized as follows. First, we cover relevant background work from a variety of fields re- lated to artificial visual aesthetics. Next, we provide an overview of our system, including the cognitive model and deep neural network approach. We then present three case studies showing initial results of our system applied to eval- uating, generating, and altering visuals for video games. We end the paper with a discussion of these results and future avenues for research. Background In this section we cover work from a set of related fields. We discuss the existing prior examples of procedural content generation applied to visual game artifacts as well as prior work in artificial visual artists. We then cover style transfer and texture generation, two related fields within the wider discipline of computer graphics, both with features relevant to our work. Visual Procedural Content Generation Visual procedural content generation, the generation of vi- sual components of a video game, has to a large extent been focused on the generation of photo-realistic textures for 3D games (Hendrikx et al. 2013). This means that the major- ity of prior work has focused on a single aesthetic (photo- realism) and a single type of visual component (textures). There are notable exceptions to this trend, including genera- tion of weapon particle effects in a space shooter (Hastings, Guha, and Stanley 2009), avatar generation for a research game (Lim and Harrell 2015), and generation of character and item art for a rouge-like RPG (Johnson 2016). Similar to our work, the game-creation and playing mobile application Gamika (Colton et al. 2016) makes use of abstract generated art, but with no communicated intention or aesthetic. Cook’s ANGELINA system stands as a particularly im- portant reference point to our work, with ANGELINA being an artificial game designer capable of expressing design in- tent and with substantial focus on aesthetics (Cook, Colton, and Gow 2016a). For example, one iteration of ANGELINA