Evaluating Educational Games Using Facial Expression Recognition Software ‐ Measurement of Gaming Emotion Mats Wiklund, William Rudenmalm, Lena Norberg, Thomas Westin, Peter Mozelius Stockholm University, Stockholm, Sweden matsw@dsv.su.se wiru3742@dsv.su.se lenan@dsv.su.se thomasw@dsv.su.se mozelius@dsv.su.se Abstract: The issue of using educational games versus entertainment games as the base for learning environments is complex, and various data to base the decision on is needed. While participant´s verbal accounts of their situation is important, also other modes of expression would be meaningful as data sources. The availability of valid and reliable methods for evaluating games is central to building ones that are successful, and should preferably include outside measurements that are less affected by the participants choice of what to share. The present study considers a method using software for analysing facial expressions during gameplay, testing its ability to reveal inherent differences between educational and entertainment games. Participants (N=11) played two games, an entertainment game and an educational game, while facial expressions were measured continuously. The main finding was significantly higher degrees of expressions associated with negative emotions (anger [p < 0.001], fear [p < 0.001] and disgust [p < 0.001]) while playing the educational game, indicating that participants were more negative towards this game type. The combination of cognitive load inherent in learning and negative emotions found in the educational game may explain why educational games sometimes have been less successful. The results suggest that the method used in the present study might be useful as part of the evaluation of educational games. Keywords: Facial expressions, emotions, games, educational games, evaluation methods 1. Introduction During recent years educational games have become an important learning tool (Eck 2006). However, using games for learning has not always been as successful as many had hoped (Ebner & Holtzinger 2007). Kirriemuir and McFarlane (2003) have identified common problems with educational games ranging from tasks that are too repetitive to games that are too simple compared to entertainment games. In some cases educational games have even seemed to combine the worst aspects of education with the worst aspects of games (Eck 2006). It is therefore critical for educational games to be both educationally sound as well as utilising core entertainment game concepts. Indeed, learning outcomes are deeply intertwined with the emotional state of the learner (Meyer & Turner 2006). Gee (2003) argues that digital games, at least those that are well designed, inadvertently are effective learning devices, both for knowledge about the game itself, but more importantly for teaching players about the world around them. There are different ways of applying digital game‐based learning, and Eck (2006) makes the distinction between using existing entertainment games in the classroom, and using games that are designed for the purpose of education. The latter games are sometimes referred to edutainment, a portmanteau for educational entertainment as they aim to be both educational as well as entertaining. An educational game can, of course, be entertaining in the sense that the player enjoys to play it, however it still aims to teach. Entertainment games do not explicitly need to have this aim. The above definition of an educational game could be taken as contrary to Gee’s (2003) definition that suggests that quite a few games could be considered educational even if it wasn’t the primary intents of the game’s creators. Gee´s definition however, is troublesome in that it makes it difficult to objectively declare a game to be educational. The only remaining objective definition is therefore a game where the creator has declared the game to be educational, which is the operational definition for the present study.