1 From MC/DC to RC/DC: Formalization and Analysis of Control-Flow Testing Criteria Sergiy A. Vilkomir and Jonathan P. Bowen, Member, IEEE Computer Society Abstract — This paper describes an approach to formalization of criteria of computer systems software testing and proposes a new Reinforced Condition/Decision Coverage (RC/DC) criterion. This criterion provides further development of the well-known Modified Condition/Decision Coverage (MC/DC) criterion and is more suitable for testing of safety-critical software where MC/DC may not provide adequate assurance. As a formal language for describing the criteria, the Z notation is selected. Formal definitions in the Z notation for RC/DC, as well as MC/DC and other criteria, are presented. Specific examples of using of these criteria are considered and some features are formally proved. This characterization is helpful in the correct understanding of different types of testing and also the correct application of a desired testing regime. Index Terms — software testing criteria, MC/DC, RC/DC, for- mal methods, Z notation. I. I NTRODUCTION Software testing criteria (or else, test data adequacy criteria or coverage criteria) play an important role of whole testing process. These criteria are used as [55]: • stopping rules that determines whether sufficient testing has been done that it can be stopped; • measurements of test quality when a degree of adequacy is associated with each test set; • generators, for test data selection. Test sets are considered as equivalent if they satisfy the same criterion. The use of testing criteria as regulatory requirements during software certification and licensing also has its own specific features and benefits. At the time of regulatory assessment, the stage of testing assessment is one of the most important where efforts of experts should be concentrated [49]. The methods and criteria of testing are traditionally divided into structural (or white-box) and functional (or black-box) aspects [35], [41]. Structural testing criteria, i.e. criteria which take into account an internal structure of the program, are in turn divided into data-flow and control-flow criteria, although the combination of the two has been considered [38], [45]. Data-flow criteria are based on the investigation of the ways in which values are associated with variables and how these associations can affect the execution of the program [55]. Control-flow criteria examine logical expressions, which de- termine the branch and loop structure of the program. This group of criteria is considered in the paper. The aim of these criteria is testing decisions (the program points at which the control flow can divide into various paths) and conditions (atomic predicates which form component parts of decisions) in a program. In the scientific literature, criteria definitions are typically informal (in natural language). Sometimes these definitions are not clear enough and this can lead to inaccurate understanding. In this paper, the task of producing formal criteria definitions is considered. As a formal language for describing the criteria, the Z notation [26], [43] is selected, which is used recently not only in academic context [5] but also for industrial development of high-integrity systems such as safety-critical software [7], [8]. This paper is based on our previous results [46], [47] and is structured as follows. Section II presents a brief review and then formal definitions of control-flow criteria. The criteria like statement coverage, decision coverage, condition coverage, decision/condition coverage, multiple-condition coverage, and full predicate coverage are addressed. Section III presents a detailed analysis of the Modified Condition/Decision Coverage (MC/DC) criterion. A definition in the Z notation is proposed and an explanation of how this formal approach can eliminate the ambiguity of informal definitions is given. A specific example using MC/DC is considered, illustrating the interdependence of the conditions and decisions. We analyze a major shortcoming of the MC/DC criterion, namely the deficiency of requirements for the testing of the “false operation” type of failures. Examples of situations when failures of this type are present are considered to illustrate the problem. These have especially vital importance for safety-critical applications in particular. To eliminate the shortcoming of MC/DC, we propose a new Reinforced Condition/Decision Coverage (RC/DC) criterion, which is considered in Section IV. Z schemas for the formal definition of RC/DC and examples of its application are provided. Some features of RC/DC as well as MC/DC are formally proved. An example when RC/DC reveals the faults, which are not revealed by MC/DC, is considered. General conclusions and directions for future work are addressed in Section V. II. CONTROL- FLOW CRITERIA A. General definitions In the testing of control-flow criteria, the concepts of ‘deci- sion’ and ‘condition’ are important. A decision is a program point at which the control flow can divide into various paths. An example of a decision is the IF-THEN-ELSE construction in Pascal and other imperative programming languages. A decision is a Boolean expression consisting of a one or several condition combined by logical connectives. A condition is an elementary Boolean expression (atomic predicate), which cannot be divided into further Boolean expressions.