Object distance and its application to adaptive random testing of object-oriented programs Ilinca Ciupa, Andreas Leitner, Manuel Oriol, Bertrand Meyer Chair of Software Engineering Department of Computer Science ETH Zurich CH-8092 Z ¨ urich {Ilinca.Ciupa, Andreas.Leitner, Manuel.Oriol, Bertrand.Meyer}@inf.ethz.ch ABSTRACT Testing with random inputs can give surprisingly good results if the distribution of inputs is spread out evenly over the input domain; this is the intuition behind Adaptive Random Testing, which relies on a notion of ”distance” between test values. Such distances have so far been defined for integers and other elementary inputs; ex- tending the idea to the testing of today’s object-oriented programs requires a more general notion of distance, applicable to composite programmer-defined types. We define a notion of object distance, with associated algorithms to compute distances between arbitrary objects, and use it to gener- alize Adaptive Random Testing to such inputs. The resulting test- ing strategies open the way for effective automated testing of large, realistic object-oriented programs. Categories and Subject Descriptors D.2.5 [Software Engineering]: Testing and Debugging—testing tools Keywords random testing, adaptive random testing, object distance, distanced- based testing 1. OVERVIEW One of the central issues of software testing is test case selection. Effective testing increasingly requires strategies for automatic test case selection, since manual selection (for which a need will al- ways remain) can only produce a subset of the large test suites that modern computing technology allows us to run. Counter to what intuition suggests, random strategies for selecting test inputs have proved remarkably effective when they can use a distribution of in- puts that is spaced evenly over the range of possible values. The family of testing strategies called Adaptive Random Testing [3] is based on this idea. They are one of the most promising directions Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. RT’06 July 20, 2006, Portland, ME, USA Copyright 2006 ACM 1-59593-457-X/06/0007 ...$5.00. of automatic test case generation. Section 2 summarizes the contri- butions of this approach. Work on Adaptive Random Testing has so far only considered inputs of primitive types such as integers, for which the notion of “even spacedness” immediately makes sense: any such input be- longs to a known interval on which there exists a total order rela- tion. To test today’s object-oriented programs, we also need to con- sider inputs that are composite objects with many fields, which ex- ist during execution as a result of instantiating classes. We feel that it is desirable to extend the attractive concept of Adaptive Random Testing to such inputs; this paper describes an adaptive random strategy for selecting objects used in test cases for object-oriented programs. The basic problem is, given a set of instances of various types, to select a subset for inclusion in a test suite so as to maximize the chances of finding a bug. Because multiple-field objects are not members of a totally ordered set, there’s no notion of “equally spaced” inputs in a range. To address this issue, we rely on a notion of object distance to determine how far an object is “spaced” from another, and use it as a basis for object selection strategies. The main contributions of this work include: • A comprehensive notion of object distance (Section 3), which takes into account the various properties of an object: its dy- namic type, which in O-O programming need not be identi- cal to the type of another object to which we need to compute the distance (as both objects might be contenders for a call to the same routine 1 in the context of dynamic binding, both of their types inheriting from a common ancestor); the values of its primitive fields (integers, reals etc.); but also the refer- ence fields that it may contain, leading to other objects and enabling the distance to take account of the structure of the run-time object graph. • Strategies for Adaptive Random Testing (Section 4) based on the object distance and making it possible to select objects from a given set so as to maximize the diversity of objects in a test suite. • A case study (Section 5), suggesting that the strategy does enhance that diversity and leads to effective testing. Section 6 discusses various issues and Section 7 draws conclu- sions and outlines directions for future work. This work is part of a general project to produce test cases en- tirely automatically on the basis of contracts equipping object-oriented 1 Another name for routines is “methods”. Throughout the paper we use Eiffel-like terminology and notations. Proceedings of the First International Workshop on Random Testing (RT’06) 55