Analyzing Interaction Orderings with Model Checking Matthew B. Dwyer, Robby, Oksana Tkachuk Kansas State University Manhattan, KS 66506, USA {dwyer,robby,oksana}@cis.ksu.edu Willem Visser RIACS, NASA Ames Research Center Moffett Field, CA 94035, USA wvisser@email.arc.nasa.gov Abstract Human-Computer Interaction (HCI) systems control an ongoing interaction between end-users and computer-based systems. For software-intensive systems, a Graphic User Interface (GUI) is often employed for enhanced usability. Traditional approaches to validation of GUI aspects in HCI systems involve prototyping and live-subject testing. These approaches are limited in their ability to cover the set of possible human-computer interactions that a system may al- low, since patterns of interaction may be long running and have large numbers of alternatives. In this paper, we propose a static analysis that is capa- ble of reasoning about user-interaction properties of GUI portions of HCI applications written in Java using modern GUI Frameworks, such as Swing TM . Our approach con- sists of partitioning an HCI application into three parts: the Swing library, the GUI implementation, i.e., code that interacts directly with Swing, and the underlying applica- tion. We develop models of each of these parts that preserve behavior relevant to interaction ordering. We describe how these models are generated and how we have customized a model checking framework to efficiently analyze their com- bination. 1 Introduction Modern software is becoming increasingly feature-rich and integrated into mission critical processes. A Graphical User Interface (GUI) serves to foster efficient and effective Human-Computer Interaction (HCI) by: (a) depicting ap- plication data in forms that allow humans to quickly and clearly understand that data, and (b) by guiding and con- trolling interaction by presenting only the set of allowable actions that users may perform next based on the under- lying application’s data state. Traditionally, GUI valida- tion has involved prototyping a graphic user interface and performing live-subject testing to assess both of these con- cerns. Live-subject testing is clearly necessary, especially in assessing usability and ergonomic aspects of GUIs, but the potentially long-lived nature of the interactions between users and system and the number of alternatives available to a user at each step in an interaction make coverage testing of the space of possible interaction orders infeasible. In this paper, we present a static approach to analyzing program behavior under all possible interaction orderings enforced by a GUI system implemented in the Java Swing framework. We discuss Swing in detail in Section 3, but it is important to understand that it presents several challenges for static analysis. Swing is an object-oriented framework where win- dows, widgets on a window (e.g., buttons, selections, text entry boxes), the text and color associated with widgets and windows, and numerous additional attributes are all defined by instantiating framework classes. References among those class instances define information about the visibility, modality, and enabledness of widgets that is cru- cial to defining the evolving state of the user-interface in response to user-initiated actions. Swing is an event-driven framework that inverts and internalizes the application’s control flow. Programmers de- fine methods, called event-handlers, that respond to occur- rences of user-initiated actions, called events, like the press- ing of a button. Handler methods are bound to events by making framework calls that record the event-handler re- lation. The framework executes a cyclic event-dispatching thread, which processes each event in turn and invokes the associated handler-methods. The fact that object values and object referencing re- lationships are used to define the structure and behavior of a Swing GUI, means that traditional static analysis ap- proaches that do not capture program data values precisely are of limited use; a more precise, object-sensitive form of analysis is needed. This observation led us to explore the use of model checking as a means of reasoning about GUI behavior in previous work [8]. In this paper, we apply soft- ware model checking approaches [5, 18, 22] to automate and scale the analysis of interaction orderings in Java GUIs to realistic systems. Our approach consists of partitioning an HCI application into three parts: the Swing framework, the GUI implementation (i.e., the GUI setup and event-handler