Model-based Replay Testing for Event-driven Software Shakaiba Majeed Department of Computer and Software Hanyang University Seoul 133-791, Korea Tel: +82-2-2220-1737 shakaiba@rtcc.hanyang.ac.kr Minsoo Ryu Department of Computer Science and Engineering Hanyang University Seoul 133-791, Korea Tel: +82-2-2220-1737 msryu@hanyang.ac.kr ABSTRACT Event-driven software systems are hard to thoroughly test because they possess a huge number of possible event sequences to test and also require significant manual effort and/or special hardware support during test case execution. In this paper we propose a novel testing approach, we call model-based replay testing that combines the advantages of model-based testing and an operating system (OS) level replay mechanism to overcome such difficulties of event-driven software testing. Specifically, we use a finite state machine (FSM) model and an n-switch set cover to automatically generate abstract test cases. We then introduce the use of OS-level replay mechanism in transforming abstract test cases into concrete test cases and executing them in an automated manner. To demonstrate the effectiveness of the proposed approach, we also describe a case study on GUI software testing using a prototype implementation of an OS- level replay mechanism in Android software platform. CCS Concepts  Software and its engineering~Software testing and debugging  Software and its engineering~Model-driven software engineering Keywords Event-driven software systems, model-based testing, automated test case generation, OS-level replay mechanism. 1. INTRODUCTION Despite the widespread use of event-driven software in modern computing systems, current practices of event-driven software testing are almost ad hoc and inefficient. This is attributed to inherent difficulties in event-driven software testing. First, event-driven systems have an enormous input space that must be taken into account during test case creation. Testers must consider a huge number of possible sequences of input events since an event-driven system exhibits different behavior with a different sequence of input events. This not only makes exhaustive testing infeasible, but also makes it difficult to create a test suite of sufficient coverage. Second, executing test cases against event-driven systems requires significant manual effort and/or special hardware support. For example, GUI software demands considerable manual effort in executing intended GUI events and verifying the results. Interrupt-driven software is another example that may require a special hardware setting to fire and test low-level I/O interrupts in a controlled manner. To address these problems, we propose a novel testing approach, we call model-based replay testing, that combines the advantages of model-based testing and an operating system (OS) level replay mechanism [10]. An important advantage of model-based testing is that test cases can be efficiently generated from a behavioral model of the system under test (SUT). In principle, if the model is formal enough, it is possible to automate the derivation of test cases without compromising on test quality and coverage. However, test cases derived from a model cannot be directly executed against the SUT since they are often described on a high-level of abstraction. Such abstract test cases thus need to be transformed into concrete test cases on a sufficiently detailed level of abstraction. This can be achieved by introducing an OS-level replay mechanism. An OS-level replay mechanism can capture and provide concrete details about low-level events, which helps us map abstract test cases to corresponding concrete test cases. Furthermore, the OS-level replay mechanism also allows us to automatically emulate any desired sequence of low-level events without any special hardware support or human intervention, thereby enabling the automated execution of concrete test cases. In this paper, we describe how to combine an OS-level replay mechanism and a model-based testing approach into an effective testing framework directed towards automated event-driven software testing. Specifically, we use a finite state machine (FSM) model and an n-switch set cover [3] for automatic generation of abstract test cases. We then introduce the use of OS-level replay mechanism in transforming abstract test cases into concrete test cases and executing them in an automated manner. To demonstrate the effectiveness of the proposed approach, we also describe a case study on GUI software testing using a prototype implementation of an OS-level replay mechanism in Android software platform. The remainder of this paper is organized as follows. Section II gives a brief overview of an OS-level replay mechanism. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. SAC 2016, April 04-08, 2016, Pisa, Italy © 2016 ACM. ISBN 978-1-4503-3739-7/16/04…$15.00 DOI: http://dx.doi.org/10.1145/2851613.2851794