Colloquium Embodying gesture-based multimedia to improve learning Chun-Yen Chang, Yu-Ta Chien, Cheng-Yu Chiang, Ming-Chao Lin and Hsin-Chih Lai Address for correspondence: Prof Chun-Yen Chang, Professor and Director, Science Education Center and Graduate Institute of Science Education and Department of Earth Sciences, National Taiwan Normal University, No. 88, Section 4, Ting-Chou Road, Taipei 11677, Taiwan. Email: changcy@ntnu.edu.tw Introduction As revealed by the recent Horizon Report (Johnson, Smith, Willis, Levine & Haywood, 2011), the creation of gesture-based interfaces (eg, Microsoft Kinect, Nintendo Wii and Apple iPhone/iPad) create promising opportunities for educators to offer students easier and more intuitive ways to interact with the content in multimedia learning environments than ever before. For instance, Kinect, a motion-sensing input device developed by Microsoft for Xbox360 (Redmond, WA, USA), enables students to use their body motions, such as swiping, jumping and moving, to interact with the content on the screen. Growing communities (eg, KinectEDucation; http://www. kinecteducation.com/) advocate integrating Kinect applications into classroom teaching to make students’ learning experiences more active and joyful. Few studies investigate the impact of gesture-based multimedia learning on students’ cognitive learning outcomes and/or its theoreti- cal underpinnings. This paper briefly discusses the theoretical underpinnings for adopting gesture-based multimedia learning, then describes how we used Kinect to embody the most common type of multimedia learning in classroom (ie, PowerPoint presentations) and finally details a preliminary study exploring the impact of gesture-based multimedia learning on stu- dents’ cognitive learning outcomes. Leveraging on human movements to facilitate multimedia learning The role human movements play in terms of learning is gaining increasing attention in the field of cognitive science. The traditional assumption that knowledge representations in cognition are composed of amodal symbols and reside in a modular semantic system is being challenged. Currently, grounded cognition rejects the traditional view that knowledge representations in cognition are independent of the modal systems for perception and action; instead, it proposes that knowledge representations in cognition are grounded in multiple ways, including simula- tions, situated action and bodily states (Barsalou, 2010). Accumulating behavioral and neural studies also suggest that bodily states can cause cognitive states as well as be effects of these states (see reviews of Barsalou, 2008; de Koning & Tabbers, 2011). The implication of grounded cognition for education is that using human movements to embody instructional materials, either demonstrations by instructors or executions by learners themselves, may enrich learners’ knowledge representations and assist them in acquiring abstract concepts. A recent meta-analysis conducted by Höffler and Leutner (2007) clearly showed that the instruc- tional efficiency of the multimedia presentations which involved human movements is generally higher than those which did not involve human movements. Several new studies also indicated that multimedia materials would be effective to assist learners in acquiring knowledge especially if they contained human movements, such as folding papers (Wong et al, 2009), tying knots British Journal of Educational Technology Vol 44 No 1 2013 E5–E9 doi:10.1111/j.1467-8535.2012.01311.x © 2012 The Authors. British Journal of Educational Technology © 2012 BERA. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA.