I I I I I I I I I I I I I I I I I I The MOTHRA Software Testing Environment· Richard A. DeMillo t Eugene H. Spafford Software Engineering Research Center Georgia Institute of Technology Atlanta, Georgia 30332-0280 + 1 404 894-3180 ABSTRACT The value of software testing in the development of large software sys- tems is well-documented. Unfortunately, the development and employment of an integrated test plan is often avoided due to the costs associated with testing. These costs include more than just capital expenses associated with obtaining test systems and software. They also include the time and effort involved in educating personnel in the use of the testing system, the time taken to run the tests, and the costs of rerunning the tests after errors are found and corrected. Furthermore, some forms of testing are difficult or impossible to run incrementally, and they produce results which may be diffi- cult to use in correcting or enhancing the tested software. The MOTHRA Environment is an integrated set of tools and interfaces that support the planning, definition, preparation, execution, analysis and evaluation of tests of software systems. The support provided by MOTHRA is applicable from the earliest stages of software design and development through the progressively later stages of system integration, acceptance test- ing, operation and maintenance. MOTHRA has been designed to address some of the cost concerns mentioned above. Two primary design criteria, in partic- ular, are significant in this regard. First, the MOTHRA interfaces-particularly user interfaces-are high-bandwidth. This allows us to present more informa- tion during testing and retesting. Coupled with proper design and integration with familiar displays, it should obviate the need for extensive training to use MOTHRA. Secondly, the overall MOTHRA architecture imposes no a priori con- straints on the size of the software systems that can be tested in the environ- ment. The practical meaning of this criterion is that the same architecture is ab!e to service programs varying in size from individual module§ of less than 10 source lines to fully integrated systems of more than 10 lines. The human user-the tester-is able to apply comparable functions across a fami- liar interface as the software being tested evolves in size and complexity by several orders of magnitude. In fact, the only indicators of size or complexity that have ties to the MOTHRA architecture are the operating system cost penal- ties and performance delays inherent in manipulating massive objects. All other costs and resource demands are under the direct control of the tester. In most cases, the tester will choose to allow critical resources such as time or memory to grow linearly with program size and complexity. The tester may, however, choose to conserve these resources by sacrificing other resources (e.g., dollars) or even by reducing the fidelity of the test. These are ulti- E. Spafford Georgia Tech 1 of 32