Analysing Engineering Expertise of High School Students Using Eye Tracking and Multimodal Learning Analytics July Silveira Gomes, Mohamed Yassine, Marcelo Worsley, Paulo Blikstein Stanford University 520 Galvez Mall, room 102 Stanford, California, 94305 {julys, myassine, mworsley, paulob}@stanford.edu ABSTRACT In this paper, we describe results of a multimodal learning analytics pilot study designed to understand the differences in eye tracking patterns found to exist between students with low and high performance in three engineering-related computer games, all of which require spatial ability, problem-solving skills, and a capacity to interpret visual imagery. In the first game, gears and chains had to be properly connected so that all gears depicted on the screen would spin simultaneously. In the second game, students needed to manipulate lines so as to ensure that no two intersected. In the final game, students were asked to position gears in specific screen locations in order to put in motion on- screen objects. The literature establishes that such abilities are related to math learning and math performance. In this regard, we believe that understanding these differences in student’s visual processing, problem-solving, and the attention they dedicate to spatial stimuli will be helpful in making positive interventions in STEM education for diverse populations. Categories and Subject Descriptors H.5.2 [Information Interfaces and Presentation]: User Interfaces. K.3.1 [Computers and Education]: Computer Uses in Education. Keywords Eye tracking, simulations, games, multimodal learning analytics, constructionism, spatial ability. 1. INTRODUCTION The need to engage and motivate more students to learn science and engineering has raise considerable awareness about Constructionist [8] and project-based pedagogies in classrooms. Understanding students’ behaviors and cognitive evolution in these open-ended environments is a challenge that is being tackled in the nascent field of Multimodal Learning Analytics [4, 10]. In particular, this study uses eye tracking to examine students’ capacity to interpret visual imagery in the context of engineering problem solving. Computer-based learning tools have become pervasive in learning environments. These technologies can be used by learners to improve their cognitive abilities and to acquire specific skills [6], including those involving visuospatial attention and perception [1]. Video and computer games habits have been shown to be related to the improvement of visuospatial abilities, including mental rotation and visual memory, even when the gender has been controlled [7]. In this paper, our interest is not in the games themselves but in the engineering, mathematical, and problem solving skills required to solve the puzzles presented in the game. Visuospatial abilities are involved in the processes of manipulating spatial forms, and these abilities are associated with different kinds of scientific thinking [13]. Performance in standardized visuospatial tasks has been associated with performance on math evaluation tests as early as primary school [5]. A study with low- and typically achieving students demonstrated that low achievers have poorer overall performance and a higher number of errors in online game-like visuospatial working memory tasks. The same study found that low achievers also demonstrated more errors and higher reaction times for arithmetic tasks [2]. Another study showed that difficulty in manipulating internal and external visuospatial representations are related to conceptual errors in chemistry, even when the problem to be solved is not explicitly spatial. These authors suggest that designing and developing tools and software to train students’ spatial visualization capacities may improve their representational and conceptual skills, which should be helpful for learning chemistry. [12]. Another researcher suggests that manual rotation is also useful to the improvement of the mental rotation skills [11]. Considering these five principles, the present effort presents data from a pilot study (n=7), where students played three online games requiring visuospatial ability in order to explore individual gaze characteristics found to be related to performance. 2. EXPERIMENTAL DESIGN Seven high school students were invited to play three online games in two separate sessions six days apart. During the first session, they played “Wheels,” a game that required them to connect gears and chains until all gears were spinning (Figure 1), as well as “Lines,” a game in which they were required to uncross lines until no intersecting lines remained. During the second session, they played “Gears,” a second gears game, the object of which was to place gears in specific locations on the screen to set on-screen objects in motion, and they also played the Lines game from the previous session. In each session, they had 5 minutes to play Wheels and Gears and 4 minutes to play Lines.