Discovering the Relationship between Student Effort and Ability for Predicting the Performance of Technology- Assisted Learning in a Mathematics After-School Program Jun Xie 1 , Xudong Huang 1 , Henry Hua 1 , Jin Wang 1 , Quan Tang 1 , Scotty D. Craig 2 , Arthur C. Graesser 1 , King-Ip Lin 1 , Xiangen Hu 1 1 University of Memphis Memphis, TN, 38152, USA {jxie2, xhuang3, hyhua, wjin, quantang, grasser, davidlin, xhu } @memphis.edu 2 Arizona State University Polytechnic, Mesa, AZ, 85212, USA scotty.craig@asu.edu ABSTRACT This study explored the relationship between students’ math ability and effort in predicting 6 th grade students’ performance in the Assessment and LEarning in Knowledge Spaces (ALEKS) system. The students were clustered into four groups by K- means: high ability high effort, high ability low effort, low ability high effort and low ability low effort. A one-way ANOVA indicated that student’s math posttest within the high ability, high effort group was significantly higher than other groups. An interaction was therefore observed between ability and effort. Further analysis revealed that math ability and effort had a multiplication impact on students’ math posttest. That is, expending effort improves student’s math posttest but how much progress in mathematics is achieved depends on the student’s math ability. Higher students’ math ability multiplies with effort in determining performance. Keywords: After-school program, ALEKS, math ability, effort, math performance. 1. INTRODUCTION Advanced learning environments [1, 4] are designed to create the most effective learning gains for students, but students vary, and not all students benefit equally from these systems. The issue then becomes discovering individual factors that maximize the usefulness of the program [9]. Studies have found that individual factors such as student effort and ability impact learning behaviors and outcomes in the computer- based learning environment [3, 8, 9]. Given the existing findings linking effort and ability with learning, this study analyzed two important characteristics on which students vary: effort and ability. We investigated the extent to which effort and math ability differentially influenced learning gains in a Web-based intelligent tutoring system (ITS) called ALEKS (the Assessment and LEarning in Knowledge Spaces) [5]. 2. METHODS Students attended the ALEKS based afterschool program two days a week for two hours a day over 25 weeks. But the learning period each day was only 1- hour, which was divided into three segments by two 20-minute breaks. Therefore the full dosage of a student was 50 hours. The 5 th grade scores of the Tennessee Comprehensive Assessment Program (TCAP) were employed to assess students’ pre-program mathematics knowledge ability. The 6th grade scores of TCAP served as the posttest scores. The data sample included 268 student volunteers from 5 middle schools in a west Tennessee (US) school district. They spent time in ALEKS between 10 hours and 40 hours. 3. ANALYSIS The independent variables were math ability and effort. Math ability was measured by TCAP 5 th grade scores. For data mining purposes, we measured student effort by the ratio of the total number of mastered topics divided by the total number of attempted topics. Although it was not a pure measure of effort, this ratio was a reliable method for contrasting student learning with persistence. The first reason was that by using artificial intelligent to map each student’s knowledge, ALEKS offered the topics he/she was ready to learn right now [2]. For students, the distances were equal between the difficulty of each student’s topics and his/her math ability. Once they seriously tried, the students would master them. Second, ALEKS required students to write down each step when solving every problem, thence students had no chance to guess the right answers to the problems. Third, this effort measure would not be contaminated by students absent –minded learning behaviors such as spent time clicking on computer screen randomly. Therefore, this ratio reflected students’ true effort in ALEKS. The dependent variables are math posttest which is TCAP 6 th grade scores. All the variables were normalized. 3.1 Clustering Math Ability and Effort The clustering goal was to divide the students into low ability high effort (LH); high ability low effort (HL); high ability high effort (HH); low ability low effort (LL). We preferred a data- driven approach that examined the data to determine the right groups. To this end, this study applied the K-means clustering algorithm with Euclidean distance based on the two attributes. The Waikato Environment for Knowledge Analysis (WEKA) was conducted to cluster the data. Table 1 showed the mean of each attribute and the number of students in each group.