International Journal of Applied Psychology 2012, 2(2): 1-7 DOI: 10.5923/j.ijap.20120202.01 Intelligent Tutoring in a Non-Traditional College Classroom Setting Elizabeth Arnott-Hill 1,* , Peter Hastings 2 , David Allbritton 3 1 Department of Psychology, Chicago State University, Chicago, IL, USA 2 Department of Psychology, DePaul University, Chicago, IL, USA 3 Department of Computing and Digital Media, DePaul University, Chicago, IL, USA Abstract Research Methods Tutor (RMT) is a web-based intelligent tutoring system designed for use in conjunction with introductory research methods courses. RMT has been shown to result in average learning gains of .75 SDs above classroom instruction in traditional college environments. However, a primary goal of the RMT project is to provide greater access to tutoring for students without access to traditional one-on-one human tutoring. Therefore, we further tested RMT’s effec- tiveness at a university that enrolls primarily non-traditional students. Although we again found evidence of RMT’s effec- tiveness, a few key outcome differences between traditional and non-traditional environments emerged, including percep- tions of the pedagogical agent and access to the system. Keywords Intelligent Tutoring, Technology and Education, Non-traditional Students 1. Introduction Computing technologies, particularly those that allow students convenient access to learning resources, are infil- trating every aspect of the educational system. Textbooks routinely come with online student resources, and many instructors regularly assign online or computer - based homework to enhance the classroom learning experience. One of these computer-based technologies, intelligent tu- toring systems (ITS), seeks to mimic the benefits of one-on-one human tutoring. Expert human tutoring is be- lieved to be among the most beneficial learning techniques. Bloom, for example, found that a successful human tutoring situation can enhance learning by up to 2.3 standard devia- tions over classroom instruction alone[1]. In contrast to more didactic approaches, such as lecture, textbook learning, or websites that mimic a traditional textbook layout, tutoring allows students to engage in a dialog that helps them to as- sess their current levels of understanding and to work coop- eratively to increase levels of knowledge[2-7]. The extent of the student’s engagement in this dialog is correlated with the student’s learning outcomes[8-10]. An ITS that uses the techniques employed in expert human tutoring, therefore, could provide access to this powerful learning technique without the drawbacks of human tutoring, which include the cost, the time involvement, and the inconvenience of coor- dinating schedules. Due to the challenges faced by many of * Corresponding author: earnott@csu.edu (Elizabeth Arnott-Hill) Published online at http://journal.sapub.org/ijap Copyright © 2012 Scientific & Academic Publishing. All Rights Reserved today’s students, including childcare, work schedules, and transportation[11], ITS have the potential to make vast con- tributions to effective teaching and learning. A number of successful ITS have been created and tested to date. For example, Koedinger and colleagues[12] con- ducted a large-scale study of the effectiveness of PAT (the PUMP Algebra Tutor) in conjunction with a high school algebra curriculum. They found that students who used the system scored 100% higher on basic skills tests than those who did not use the system. In laboratory-based tests, an- other ITS, AutoTutor, increased students’ scores by one standard deviation above reading a textbook alone[13]. ANDES, an ITS used in undergraduate physics courses, has also been associated with significant learning gains for stu- dents who use the system in lieu of pencil-and-paper based homework[14]. Research Methods Tutor (RMT) is a dialog-based ITS created to increase student learning in introductory under- graduate psychology research methods courses. Most psy- chology programs require at least one course in research methods[15]. Research methods courses, however, tend to be difficult for psychology majors due to their applied, techni- cal, and largely quantitative nature. In addition, research methods courses require students to engage in critical thinking, which is generally a skill that undergraduate col- lege students are in the process of developing[16]. RMT is designed for use in conjunction with traditional coursework and is available online, making it convenient to use at any time of the day without cost. It consists of five topic modules that coincide with typical topics from psychology research methods - ethics, variables, reliability, validity, and experi- mental design. Each RMT topic module is assigned after the