Taking a signal: A review of gesture-based computing research
in education
Feng-Ru Sheu, Nian-Shing Chen
*
Department of Information Management, National Sun Yat-Sen University 70, Lienhai Rd., Kaohsiung, 80424, Taiwan
article info
Article history:
Received 31 October 2013
Received in revised form
12 June 2014
Accepted 16 June 2014
Available online 24 June 2014
Keywords:
Humanecomputer interface
Interactive learning environments
Interdisciplinary projects
Pedagogical issues
Teaching/learning strategies
abstract
This study used content analysis of journal articles from 2001 to 2013 to explore the characteristics and
trends of empirical research on gesture-based computing in education. Among the 3018 articles retrieved
from 5 academic databases by a comprehensive search, 59 articles were identified manually and then
analyzed. The distribution and trends analyzed were research methods, study disciplines, learning
content, technology used, and intended settings of the gesture-based learning systems. Furthermore,
instructional interventions were also analyzed based on the learning context or the sub-education
domain to which they belonged to ascertain if any instructional intervention was applied in these sys-
tems. It was found that experimental design research is the most commonly used method (72.9%) fol-
lowed by design-based research (20.3%). The findings indicate that Nintendo Wii is the gesture-based
device that is the most often used (40%), while the domain in which the technology is most frequently
used is special education (42.4%). The same trend was also found in a further analysis which identified
that the domain which uses Wii the most is special education (70%). Among all the identified learning
topics, motor skills learning has the highest percentage (44%). When grouping these topics into three
domains of knowledge (procedural, conceptual, and both), the result demonstrates that both procedural
and conceptual type of knowledge are equally distributed in the gesture-based learning studies. Finally, a
comparison of instructional intervention of gesture-based learning systems in different sub-education
domains is reported.
© 2014 Elsevier Ltd. All rights reserved.
1. Introduction
As revealed by the Horizon Report (Johnson, Adams, & Cummins, 2012; Johnson, Levine, Smith, & Stone, 2010; Johnson, Smith, Willis,
Levine, & Haywood, 2011), gesture-based devices as an emerging technology have opened up new opportunities for learning. The fea-
tures of gesture-based devices allow the user as a controller to interact with the computer more directly through the use of motions and
movements as naturally as in daily life (Johnson et al., 2012), such as by using speech, gestures, body movements, finger flips and even facial
expressions and eye movements (Johnson et al., 2012, 2011; Wojciechowski, 2012). For example, DephJS from MIT allows users to interact
with the Google Chrome web browser through gestures. Other examples such as the 3Gear System, MudPad, LZI Technology, and ZeroTuch
(Johnson et al., 2012) also allow users to interact with computers through gestures and hand movements.
Although initially gesture-based computing received great attention in gaming and in mobile devices, the potential for learning purposes
has recently generated enormous interest among educators. The applications and development of gesture-based computing in training and
education are continually expanding. Experiments and innovative teaching with these devices have been growing in many areas such as
special education, physics, mathematics, physical therapy, arts, music, science, literacy, and social development (see the 2012 Horizon
Report for a review). Educational researchers are not only interested in investigating the effects of gesture-based devices as a means of
natural input, but also in the impact and effect it may have on other aspects of learning, such as memory (Chao, Huang, Fang, & Chen, 2013)
and physical rehabilitation (Chang, Chen, & Huang, 2011). A sampling of exciting projects for the application of gesture-based computing
* Corresponding author. Tel.: þ886 988660445.
E-mail addresses: fsheu711@gmail.com (F.-R. Sheu), nschen@mis.nsysu.edu.tw, nianshing@gmail.com (N.-S. Chen).
Contents lists available at ScienceDirect
Computers & Education
journal homepage: www.elsevier.com/locate/compedu
http://dx.doi.org/10.1016/j.compedu.2014.06.008
0360-1315/© 2014 Elsevier Ltd. All rights reserved.
Computers & Education 78 (2014) 268e277