Self-Predicted and Actual Performance in an Introductory Programming Course Paul Denny Andrew Luxton-Reilly John Hamer Department of Computer Science The University of Auckland Private Bag 92019 Auckland, New Zealand Dana Dahlstrom Department of Computer Science and Engineering University of California, San Diego 9500 Gilman Drive La Jolla, CA, USA Helen Purchase Department of Computing Science University of Glasgow Glasgow, Scotland ABSTRACT Students in a large introductory programming course were asked twice to predict their scores on the final exam: once at the be- ginning of a six-week module, and once at the end. In between, students in only one of the two lecture streams recorded subjec- tive confidence in their answers to individual questions on weekly quizzes. Students’ predictions were moderately correlated with their scores. Students who attended more quizzes had not only higher exam scores, but improved their predictions more than those who attended fewer quizzes. Practice recording confidence on indi- vidual quiz questions did not yield significantly more improvement in exam predictions. Several findings from previous work are con- firmed, including that women were significantly more underconfi- dent than men. Categories and Subject Descriptors K.3.2 [Computers and Education]: Computer and Information Science Education—Computer science education,Self-assessment General Terms Experimentation, Human Factors, Measurement, Performance Keywords learning, metacognition, confidence, gender 1. INTRODUCTION The ability to self-assess what one does and does not know is a metacognitive skill that plays a key role in learning through, for example, allocation of study time [15, 2]. Research has shown less-skilled learners not only make more errors, but are less able to distinguish when their answers are and are not correct [11]. Ear- lier work has suggested a learner’s “feeling of knowing” can be Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ITiCSE 2010 Bilkent, Ankara, Turkey Copyright 2010 ACM X-XXXXX-XX-X/XX/XX ...$10.00. made more accurate and that it would be worthwhile to develop techniques for doing so [16]. Investigators in metacognition have distinguished micropredic- tions, in which subjects judge how likely they are to be correct on each question, from macropredictions of overall performance [20]. The main question motivating the work we report here is whether, in the context of an introductory programming course, students who regularly recorded micropredictions during quizzes would im- prove their ability to make macropredictions of their final-exam performance or to answer the quiz or exam questions correctly. Ancillary to the main thrust, we also investigated other basic questions: how informative and well-calibrated students’ micro- and macropredictions were, and how quiz attendance (likely to re- flect lecture attendance in general) affected exam scores and macro- predictions. Previous work showed perceived self-efficacy at the end of a pro- gramming course was better correlated to performance than was self-efficacy at the beginning [22]; we also generally expected pre- dictions to be better at the end. Previous work also suggests women are less confident (and less overconfident) than men in computer- science courses specifically [1, 9], and in general, particularly for tasks perceived as congruent to male sex roles [17, 13]. Accord- ingly we expected similar findings regarding gender. Our research questions can be summarised as follows: 1. Quiz micropredictions: When students are more confident on an item, are they more likely to be correct? How informative and well calibrated are micropredictions, and does this vary by gender or quartile? 2. Exam macropredictions: How did students’ late exam pre- dictions differ from their early predictions, and how well did each predict actual exam scores? 3. Attendance and performance: What is the relationship be- tween the number of quizzes attempted and exam scores? Between quiz scores and exam scores? 4. Effects of micropredictions: Does recording item-specific confidence estimates on weekly quiz questions improve per- formance, or macropredictions of exam performance? 2. METHOD The Engineering Computation and Software Development course (ENGGEN 131) is compulsory for all first-year engineering stu- dents at the University of Auckland. ENGGEN 131 consists of a