Copyright 2003 Chillarege Press Fast Abstract ISSRE 2003 An Industrial Case Study: Using CACC to Test Real-time Embedded Software 1 Jing Guan and Jeff Offutt ISE Department George Mason University Fairfax, VA 22030-4444 USA { jguan, ofut }@gmu.edu 1 Certain details are omitted from this fast abstract to protect the company’s confidentiality. 1. Introduction Correlated Active Clause Coverage (CACC) is a test criterion whose goal is to test individual clauses within logical expressions [1]. Although considered to be expensive, the FAA requires its close cousin, Multiple Condition Decision Coverage (MCDC), to be used to test safety-critical parts of aeronautics software [3]. This fast abstract presents results from a case study of CACC on a safety-critical, real-time embedded transportation control system. Our study compared CACC testing with functional testing. We found that the CACC test cases detected important faults that were not detected by functional testing, and that would be very difficult to detect by any other testing technique. 2. Background Generally speaking, functional testing derives test cases from the software requirements specification. The functional tests were run on the target hardware without the use of simulation. CACC testing designs test cases based on the structure of the source code. Each clause must be shown to individually determine the value of the predicate outcome. It is equivalent to the masking form of MCDC [2]. The control system is a very complicated collection of interacting state machines and algorithms that was recently completed and deployed. It interacts with a number of hardware devices, including a battery, generator, and several sensors. It is written in C and consists of 12 files and 90 methods, which collectively contain 70 predicates. 50 predicates had one clause, 17 predicates had two, and 3 predicates had three clauses. These predicates resulted in 134 CACC tests. The software used was a complete, though pre-production version. 3. Experiment and Results The CACC tests were run on the target hardware in a simulation environment, which provided the software with inputs and collected output messages and results. Each test case was run via a test script that sets the initial conditions of the system, calls the simulation program, launches the control system, and displays the results. Several additional routines were written to allow the tester to choose test cases. These routines allow the tester to start the control system in a particular mode, choose the next test case, and operate the unit in normal mode. The test scripts included everything necessary for the test cases to be easily implemented, run and repeated. A test engineer performed the functional testing directly on the target hardware. Simulation was not used and the results were checked by hand. Eight faults were found. The CACC tests were able to find eleven additional faults, plus three of the previous faults. Two faults were related to multi-clause predicates and are very unlikely to have been found by any criterion less stringent than CACC. Two faults were in the software design, one could not be found without simulation, and six required analysis of outputs that were difficult or impossible to observe without simulation. Two faults were a result of late changes in the software that were not implemented in all locations. The first is outlined here. In normal operation, the unit goes to different modes based on two inputs—here called I1 and I2. A specific value for variable X puts the unit into mode M1, while a value of another variable Y puts the unit into mode M2. The following decision encodes this operation: Decision Block 1 : if (X >= 50) modeM1();