Using Cloze Procedure Questions in Worked Examples in a Programming Tutor Amruth N. Kumar Ramapo College of New Jersey 505 Ramapo Valley Road, Mahwah, NJ 07430, USA amruth@ramapo.edu Abstract. In order to increase the engagement of learners, we incorpo- rated cloze procedure questions into the worked-example-style feedback provided by problem-solving tutors currently used by introductory pro- gramming students unsupervised. We conducted a multi-institution con- trolled study to evaluate the effectiveness of this intervention from fall 2012 through spring 2014. The results of the study were mixed. We found that when students had to answer cloze procedure questions embedded in the feedback, they did spend significantly more time per problem and they learned concepts with significantly fewer practice problems. How- ever, they did not learn significantly more concepts and their change in score from pretest to post-test was not any different on the learned con- cepts from that of control group. Finally, the increased time on task due to the intervention may benefit different demographic subgroups differ- ently. Keywords: Cloze procedure questions, Worked example, Programming tutor, Evaluation 1 Introduction A typical software tutor provides feedback designed to help students learn from mistakes. But when students use a tutor after hours, unsupervised, on their own time, and as part of a class assignment for which they get completion credit, but not necessarily credit for improvement in learning, (conditions henceforth referred to as in-natura), how can we encourage students to read the feedback? How can we ensure that they indeed read the feedback? One mechanism is to have the students answer questions embedded within the feedback. If students are required to answer these questions before moving on to the next problem, and furthermore, required to answer them correctly, they would have to read the feedback to understand the context before answering the questions. This is the spirit in which we incorporated questions into the worked-example-style feedback [1] provided by the problem-solving tutors we have developed for programming concepts, that are typically used in-natura by dozens of schools each semester. The worked-example-style feedback itself has been shown to improve learning in our tutors in prior evaluations [2].