Performance evaluation of an online argumentation learning assistance agent Chenn-Jung Huang a, * , Yu-Wu Wang a , Tz-Hau Huang a , Ying-Chen Chen a , Heng-Ming Chen b , Shun-Chih Chang a a Department of Computer Science and Information Engineering, National Dong Hwa University, No.1, Sec 2, Da Hsueh Rd. Shoufeng, Haulien 97401, Taiwan, ROC b Institute of Electronics Engineering, National Dong Hwa University, No.1, Sec 2, Da Hsueh Rd. Shoufeng, Haulien 97401, Taiwan, ROC article info Article history: Received 23 February 2010 Received in revised form 19 January 2011 Accepted 21 January 2011 Keywords: Cooperative/collaborative learning Elementary education Evaluation methodologies Evaluation of CAL systems Intelligent tutoring systems Learning assistant State-transition diagrams Online argumentation Heterogeneous grouping abstract Recent research indicated that students’ ability to construct evidence-based explanations in classrooms through scientific inquiry is critical to successful science education. Structured argumentation support environments have been built and used in scientific discourse in the literature. To the best of our knowledge, no research work in the literature addressed the issue of automatically assessing the student’s argumentation quality, and the teaching load of the teacher that used the online argumentation support environments is not alleviated. In this work, an intelligent argumentation assessment system based on machine learning techniques for computer supported cooperative learning is proposed. Learners’ arguments on discussion board were examined by using argumentation element sequence to detect whether the learners address the expected discussion issues and to determine the argumentation skill level achieved by the learner. Learners are first assigned to heterogeneous groups based on their responses to the learning styles questionnaire given right before the beginning of learning activities on the e-learning platform. A feedback rule construction mechanism is used to issue feedback messages to the learners in case the argumentation assessment system detects that the learners go in a biased direction. The Moodle, an open source software e-learning platform, was used to establish the cooper- ative learning environment for this study. The experimental results exhibit that the proposed work is effective in classifying and improving student’s argumentation level and assisting the students in learning the core concepts taught at a natural science course on the elementary school level. Ó 2011 Elsevier Ltd. All rights reserved. 1. Introduction The recent research indicated that students’ capability of constructing the explanations based on evidence in classrooms through scientific inquiry is critical to successful science education (Kuhn & Resier, 2005; Lemke, 1990; National Research Council [NRC], 1996; Rosebery, Warren, & Conant, 1992; Schauble, Glaser, Duschl, Schulze, & John, 1995). Researchers presented that argumentation is directly related to science education because students show their understanding of science concepts through scientific inquiries (Jimenez- Aleixandre, Rodriguez, & Duschl, 1999; McAlister, 2001). Nussbaum and Sinatra (2003) suggested that teachers look for different tools to promote knowledge restructuring in their students, and argumentation indeed has a possibility as a conceptual change intervention. Simon, Erduran, and Osborne (2006) presented their work on developing argumentation in school science education, and they recognized that the opportunity for students to discuss and argue if their evidence supported a theoretical explanation or not was helpful to students’ involvement in scientific thoughts. PISA 2006 Science Competencies for Tomorrow’s World specifically indicated that students should be able to construct a logical argument for a hypothesis by using data from diverse sources (Organisation for Economic Cooperation and Development [OECD], 2007). It was claimed that practice of argumentation is an essential teaching activity in science education (Driver, Asoko, Leach, Mortimer, & Scott, 1994; Duschl & Osborne, 2002; Sandoval & Millwood, 2005; Zohar & Nemet, 2002). A complete argu- mentation analysis included several aspects of knowledge including linguistic constraints, domains correlation, conceptual relations and discourse structure. Osborne, Erduran, and Simon (2004) indicated that the promotion of the argumentation level surely promotes students’ thinking and reasoning skill, and it also develops their skills of estimation and judgment. * Corresponding author. Tel.: þ886 38633503; fax: þ886 38634010. E-mail address: cjhuang@mail.ndhu.edu.tw (C.-J. Huang). Contents lists available at ScienceDirect Computers & Education journal homepage: www.elsevier.com/locate/compedu 0360-1315/$ – see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2011.01.013 Computers & Education 57 (2011) 1270–1280