Helping Students Test Programs That Have Graphical User Interfaces Matthew Thornton, Stephen H. Edwards, and Roy Patrick Tan Dept. of Computer Science, Virginia Tech 660 McBryde Hall, Mail Stop 0106 Blacksburg, VA 24061, USA +1 540 231 5723 { thorntom, edwards, rtan}@cs.vt.edu ABSTRACT Within computer science education, many educators are incorporating software testing activities into regu- lar programming assignments. Tools like JUnit and its relatives make software testing tasks much easier, bringing them into the realm of even introductory stu- dents. At the same time, many introductory pro- gramming courses are now including graphical inter- faces as part of student assignments to improve stu- dent interest and engagement. Unfortunately, writing software tests for programs that have significant graphical user interfaces is beyond the skills of typical students (and many educators). This paper presents initial work at combining educationally oriented and open-source tools to create an infrastructure for writ- ing tests for Java programs that have graphical user in- terfaces. Critically, these tools are intended to be ap- propriate for introductory (CS1/CS2) student use, and to dovetail with current teaching approaches that in- corporate software testing in programming assign- ments. We also include in our findings our proposed approach to evaluating our techniques. Keywords: on-line education, computer science, test- driven development, test-first coding, GUI, ob- jectdraw, JUnit 1. INTRODUCTION With the loss of productivity and system downtime caused by code defects, it is important for information technology students to learn how to test software. At the same time, however, lack of educational support for testing and quality assessment in university computer science curricula can result in students that are ill prepared for producing commercial-quality code. As a result, it is becoming more common for computer science edu- cators to include software testing activities across multiple courses, often by adding software testing requirements to tradi- tional programming assignments. Approaches include using explicit instructor-provided tests in assignment specifications, using instructor-provided tests for automatic grading, requiring students to write test plans and test cases, and even requiring students to practice test-driven development (TDD). Recent studies using test-driven development (TDD) in the classroom show that students produce higher quality code when they write their own tests, with a 28% reduction in the number of bugs per thousand lines of student-written code (KSLOC), on average [14]. In fact, when students were required to write their own tests and were graded on how well they did this using our tech- niques, the top 20% of students in our most recent experimental evaluation achieved defect rates of approximately 4 defects per KSLOC or better, which is comparable to most commercial- quality software written in the United States. Of the students in the control group who were not required to turn in their own tests and were not evaluated on their own testing behavior, none achieved this level of performance, with the best score reaching only 30 defects/KSLOC [14, 16]. Consequently, it has been demonstrated that test-driven development has some impact on the quality of student-written code. Testing frameworks, such as JUnit for Java [3, 22] and similar XUnit frameworks for languages such as C++ [7] are a critical enabling factor in developing a curriculum around test driven development. Many educators have found that JUnit makes writing tests easy, even for introductory-level students. Most modern interactive development environments for Java, includ- ing those targeted at educational communities, offer student- friendly support for JUnit. The spread of JUnit as the de facto standard for writing unit-level tests in Java has provided a use- ful educational advantage in this regard. At the same time, however, it is also becoming more common for introductory programming courses to include graphical user interface (GUI) aspects in assignments. GUIs are a common metaphor used in discussing object-oriented programming tech- niques [10]. GUIs also aid in explaining basic programming concepts, because activities such as implementing the “what happens next” response to a mouse click or what happens when you drag and drop an item into a bin can be quickly understood by beginning programmers. This, in addition to the prolifera- tion of GUI frameworks available such as Swing [5] and ob- jectdraw [11], makes teaching students to program GUIs a very inviting prospect for instructors. However, while there are a number of level-appropriate educational GUI frameworks to simplify teaching tasks, there is no level-appropriate support for testing GUI applications. This dilemma is illustrated in Figure 1. Consequently, it is necessary to develop a framework that allows introductory computer science students to develop test cases for their GUI-based programming assignments if one wishes to include software testing activities. To address this problem, we are adapting an introductory GUI package called objectdraw [11], together with the Abbot GUI testing library [1, 13] (based on JUnit), to develop a student- friendly testing framework for GUI assignments. These tools can be extended to make student testing easier and can be inte-