Empirical Evaluation of a UML Sequence Diagram with Adornments to Support Understanding of Thread Interactions Shaohua Xie 1 , Eileen Kraemer 1 , and R.E.K. Stirewalt 2 University of Georgia 1 , Michigan State University 2 { shaohua, eileen }@cs.uga.edu, stire@cse.msu.edu Abstract Programs that use multi-threaded concurrency are known to be difficult to design. Moreover, research in computer-science education suggests that concurrency and synchronization concepts are generally difficult to master. It stands to reason that comprehension tasks may be more complex for programs that employ con- currency than for sequential programs. We believe that external representations, specifically refinements to some of the popular UML modeling notations, should aid students in mastering fundamental concur- rency/synchronization concepts and should enable practitioners to better comprehend the dynamically evolving nature of the these programs. In this paper, we present our synchronization adorned UML (saUML) sequence diagram notation that highlights aspects of thread interactions and describe an empiri- cal study of whether these diagrams, as opposed to purely textual representations, help students to better understand concurrent executions and concurrency concepts, as measured by their ability to answer ques- tions about a particular execution of a multi-threaded system. A statistically significant benefit was found from the study. 1. Introduction Programs that use multi-threaded concurrency are known to be difficult to design. Moreover, research in computer-science education suggests that concurrency and synchronization concepts are generally difficult to master [3, 4, 6, 11, 14]. It stands to reason that com- prehension tasks may be more complex for programs that employ concurrency than for sequential programs. This expectation is consistent with our experience [1, 12] and the experiences of others [6, 7, 10]. Many of the difficulties that attend to concurrent- program comprehension derive from the complexities that make concurrency and synchronization difficult for students to learn in the first place. For example, the space of thread interleavings is enormous and difficult to meaningfully visualize in its entirety. When threads synchronize, they do so indirectly, using calls to low- level system primitives that modify the state of the op- erating system’s data structures. That students find these primitives difficult to reason about is well docu- mented [4, 8, 9]. We believe external representations [16] can both aid students in mastering concurrency and synchroniza- tion concepts and enable practitioners to better com- prehend the dynamically evolving nature of concurrent programs. To this end, we have been conducting em- pirical studies to assess the effectiveness of various representations on human performance (and retention of knowledge) during comprehension tasks involving concurrent programs. This paper describes a study, conducted at the University of Georgia in 2006. The study shows that a modest extension to the UML 2.0 sequence diagram notation [13], when used in conjunc- tion with source code, can significantly improve under- standing tasks that involve specific executions of a con- current program when compared with using only the source code. The improvement was measured by user performance in answering scenario-based questions about those executions. The remainder of the paper is structured as follows. We first outline some of the fundamental problems that students face when learning about concurrency and synchronization and describe prior work in the use of external representations to improve learning (Section 2). We then describe our synchronization adorned UML (saUML) sequence diagram notation (Section 3), the experimental design, and the results of the study itself (Section 4), and the conclusions we were able to draw from this work (Section 5). 2. Background Several studies have looked at how students learn about concurrency and the kinds of problems that at- tend to this learning. Kolikant’s research shows that