Predictive Student Modeling for Interventions in Online Classes Michael Eagle TutorGen, Inc. 1037 S. Fort Thomas Ave Fort Thomas, KY 41075 MichaelJohnEagle@gmail.com Mary Jean Blink TutorGen, Inc. 1037 S. Fort Thomas Ave Fort Thomas, KY 41075 mjblink@tutorgen.com Ted Carmichael TutorGen, Inc. 1037 S. Fort Thomas Ave Fort Thomas, KY 41075 ted@tutorgen.com John Stamper Carnegie Mellon University 5000 Forbes Ave. Pittsburgh, PA 15213 jstamper@cs.cmu.edu Jessica Stokes Western Governors University 4001 South 700 East, ste. 700 Salt Lake City, UT 84107 jessica.stokes@wgu.edu Jason Levin Western Governors University 4001 South 700 East, ste. 700 Salt Lake City, UT 84107 jason.levin@wgu.edu ABSTRACT As large-scale online classes become more prevalent there is great interest in finding ways to model students at scale in these classes in order to predict outcomes. Student models, if successful, would help determine strong predictors of student success, which would highlight potential causal factors for such success, allowing schools to focus on refinements and interventions that positively impact their student outcomes. In this research, TutorGen has partnered with Western Governors University (WGU), a large online university, and gathered data at scale in order to build exploratory models to predict student outcomes. This paper presents our results so far in successfully identifying students who will pass (or even take) the final exam. We have examined the order in which students take courses, as well as the timing of starting and completing work; our initial analysis reveals that these are strong predictors of course outcomes. Keywords Predictive modeling, online education, student interventions, cognitive models, student models, online courseware, feature selection, data visualization, mixed effects modeling, logistic regression. 1. INTRODUCTION In collaboration with Western Governors University, a large online university, we have been examining large data sets of students’ online interactions, which will help provide insight into the way students succeed in course completion. Our initial work has focused on building a set of predictive models looking at success within course and also between courses. Based on student in course data as well as post course assessment, we know learning is occurring within courses. We do know that some students do not pass the final assessment, and these are the students we were most interested in modeling. From our exploratory data analysis and initial models, three important and distinct factors emerged. First, there was a “basic dropout,” students who did not pass simply because they stopped being active in the course – not completing assignments or taking tests. Second, was a group termed “late and out of time” that started very late in the semester and appeared to run out of time. And finally, there was group we termed “exam avoidance” that appeared to have mastered enough to pass the final assessment, but for unknown reasons did not attempt the test. We examined some of the properties of each of these types of students and suggest strategies that we plan to implement in order to intervene with each type appropriately. 2. BACKGROUND Western Governors University (WGU) was founded in 1997 by nineteen governors as a non-profit, competency-based, 100% online university, which has graduated more than 100,000 students. WGU currently has more than 94,000 students across all 50 states. Offering 60 degrees in four colleges supporting high- demand fields such as business, K-12 teacher education, information technology, health professions, WGU’s success is founded on their unique learning ecosystem that is student- centric, competency-based, and 100% technology enabled. This approach lowers tuition and provides faster time to graduation. The WGU competency-based model enables students to leverage their existing knowledge and skills while seeking to improve their opportunities to advance their careers. Students earn college credit by demonstrating what they know and can do rather than basing the credit on seat-time in a course. The curriculum and assessments are defined by career-relevant competencies to accelerate learning according to a student’s level of experience. WGU provides the curriculum, formative assessments, and summative evaluation. In addition, the WGU student-centric support services promote success in student learning through disaggregated faculty roles including: program mentors, course instructors, evaluators, and curriculum and assessment developers. This approach allows students to remain at the center of all activities such that the focus is on the learning of each student. The success of the online model that combines competency-based learning models with strong student support is demonstrated by their high student satisfaction ratings, contributing to a higher than average retention rate. WGU is committed to continually evaluating all aspects of their educational experience in order to enhance their models, content, delivery methods, and student support through proven academic research. This approach helps them make strategic and impactful improvements to student learning outcomes while enhancing the overall student experience (reduced time to achievement, career advancement, learning support, completion/achievement etc.) This drive to measure and identify areas for improvement resulted in a deep data dive evaluating courses that have relatively lower completion rates than others within the same program.