Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2015 2015 Paper No. 15105 Page 1 of 14 Mars Game: Creating and Evaluating an Engaging Educational Game Kevin Dill Barbara Freeman Spencer Frazier Juan Benito Lockheed Martin Mission Systems & Training Standards Work / UC Berkeley Lockheed Martin Mission Systems & Training Cooperative Entertainment Inc. Burlington, MA Quincy, MA Burlington, MA Raleigh, NC kevin.dill@lmco.com bfreeman@berkeley.edu spencer.j.frazier@lmco.com royston.benito@gmail.com ABSTRACT Games have been studied for some time as a possible supplement to classroom-based learning and training, yet questions remain about how best to create the content. How can learning and gaming be merged in a way that does not diminish the positive aspects of either? Games that poorly integrate the educational component suffer for failing to teach, and games that poorly design the entertainment component suffer because they fail to engage the student. The content, and its comprehension by students, must then be evaluated to determine whether learning actually occurred. This paper describes the design, implementation, and evaluation of a game – the Mars Game prototype – that has been crafted from the beginning with the intent of emphasizing the educational content and the player’s engagement (i.e. the “fun”) in equal measure. The game teaches 9th and 10 th grade math and programming concepts, and aligns to the Common Core State Standards for Mathematics. The results of a randomized control study performed with students from a U.S. high school are provided. The study evaluated the effect of the Mars Game prototype on students learning and engagement. The study demonstrated that the game statistically significantly improved learning outcomes against a comparison group. The study also showed that the treatment group scored significantly higher than the control group on engagement and deep immersion in the gameplay. These evaluations provide encouraging results reinforcing the design goals set out for the game as well as the efficacy of game-based learning – when you have a truly engaging game. ABOUT THE AUTHORS Kevin Dill is a member of the Group Technical Staff at Lockheed Martin Mission Systems and Training and the Lead Developer on the Mars Game project. He is a veteran of the game industry with seven published games (including two Zoo Tycoon 2 expansions, Iron Man, and Red Dead Redemption). Kevin was the technical editor for Introduction to Game AI and Behavioral Mathematics for Game AI, and a section editor for AI Game Programming Wisdom 4 and the Game AI Pro series. He is a prolific author and speaker, and has taught at Harvard University, Boston University, and Northeastern University. Barbara Freeman Ed.D, is the principal researcher on the Mars Game project. She is a visiting scholar at U.C. Berkeley, and currently a researcher of digital games for the Advanced Distributed Learning Colab of the Department of Defense and the World Bank amongst others. Barbara has over 25 years of domestic and international experience in technology, education, research and development, analytics, and risk management, and has created K-12 educational interventions and established products for Blue Chip clients and government agencies globally. Spencer Frazier is a researcher, developer, and designer with multiple publications investigating the intersection of games and AI, machine learning, multi-agent systems, and human computation, a software engineer at Lockheed Martin Mission Systems and Training, and a developer on the Mars Game project. He has notable publications at AAAI and AIIDE for his work with a location-based mobile game studying human and artificial agent interaction in real and simulated environments. Previous projects he contributed design, development and data analysis for include: Team-It: A Multi-agent real-world system simulation and research project and Snapets: A human computation game with a purpose.