The Development of an Intelligent Game-Based System for Learning Sign Language with Kinect Teerawat Kamnardsiri, Ler-on Hongsit, Pattaraporn Khuwuthyakorn and Noppon Wongta College of Arts, Media and Technology, Chiang Mai University, Chiang Mai, Thailand teerawat.k@cmu.ac.th ler-on_h@cmu.ac.th pattaraporn.k@cmu.ac.th noppon.w@cmu.ac.th Abstract: Sign Language (SL) is not a universal language, but diverse sign languages have been used in several countries, such as the UK, the US and Thailand. Sign Language is a visual language for communication between people with hearing impairment in everyday life. Nonetheless, standard sign language experts are rare and there is a lack of experts in the specification of standard sign languages. In this paper, we present the development of an intelligent game-based system for learning sign language with Kinect aiming at the practice of sign language for students with hearing impairment. The game-based learning provided learning activities that could suggest students to achieve an end goal. Moreover, knowledge capturing was employed by the knowledge engineering method from three sign language experts, as well as an action of sign language. After further analysis, immediate feedback and suggestions based on knowledge transfer from American Sign Language (ALS) experts were given. This experimental divided the students (N=31) into two groups. Group A (N=17) learnt with game-based learning while Group B (N=14) learnt with traditional face to face learning. As a result, this study provided a significant difference (p<0.05) on the mean score of the post-test of both Group A as well as Group B. The final result showed that the game-based learning approach provides a performance of ASL vocabularies higher than the traditional face to face learning approach. Keywords: Game-based learning; sign language; recognition system; kinect; expert system; knowledge engineering 1. Introduction Recently, sign language in deaf school classrooms was taught with traditional face to face learning approach as shown in Figure 1 (c). Then total communication such as listening, speaking, lip-reading, writing, reading and sign language were employed to improve the communication of hearing-impaired students (Kamnardsiri et al., 2009). However, there is a lack of teachers with professional Sign Language (SL) skills for teaching hearing- impaired students. SL is a visual language for communication between people with hearing-impairment in everyday life. SL consists of finger spellings and signs. Finger-spellings are a mixture of continuous hand configurations; whereas, signs are gestures of continuous hand motions and hand configurations. Due to the limited budget available to support the deaf study, such as software development, electronic tools, and game- based learning, learning sign language in deaf schools is quite complicated and requires more time to transfer knowledge skills for the students. In order to communicate with foreign people with hearing-impairment, students need to know a standard sign language, such as American Sign Language (ASL). As a result, there needs to be some tools or systems to support their learning of American Sign Language (ASL). Computer games, especially education games or serious games, have provided significant achievements for learning. Until now, most Game-Based Learning (GBL) researches have focused on the effectiveness of learning such as strategies, learning achievements, motivation, engagement and behavioral patterns (Tsai, Huang, Hou, Hsu, & Chiou, 2016). There are several effective researches related to game-based learning; for example, Papastergiou investigated potential gender differences in the game-based learning effectiveness and motivational appeal of Greek high school Computer Science (CS) curriculum (Papastergiou, 2009). Moreover, Cheng and Su developed a game-based learning system to improve self-efficacy for student’s learning. They mapped the course content into the game to provide a scenario environment of learning (Cheng and Su, 2012). Furthermore, Lester and his collogues designed game-based learning named Crystal Island: Uncharted Discovery to learn environment for upper elementary science education (Lester et al., 2014). Additionally, Hsiao and Chen developed a gesture interactive game-based learning (GIGL) approach to improve preschool children (Hsiao and Chen, 2016). With all the mentioned information above in mind, for this study, we present the development of an intelligent game-based system for learning sign language with Kinect that aims to practice sign language for students with hearing impairment. The game-based learning provides learning activities that can suggest students to achieve an end goal. Moreover, the game-based learning can also assist teachers for sign language improvement of their students. 304