Modeling Software Testing Processes James S. Collofello Zhen Yang, John D. Tvedt Derek Merrill, Ioana Rus Computer Science and Engineering Department Arizona State University Tempe, Arizona 85287 (602) 965-3190, collofello@asu.edu Abstract The production of a high quality software product requires application of both defect prevention and defect detection techniques. A common defect detection strategy is to subject the product to several phases of testing such as unit, integration, and system. These testing phases consume significant project resources and cycle time. As software companies continue to search for ways for reducing cycle time and development costs while increasing quality, software testing processes emerge as a prime target for investigation. This paper proposes the utilization of system dynamics models for better understanding testing processes. Motivation for modeling testing processes is presented along with a an executable model of the unit test phase. Some sample model runs are described to illustrate the usefulness of the model. Motivation Software testing activities consume a significant amount of project resources and development cycle time. Typical testing costs vary based on the criticality of the project from 40% to 85% of the development life cycle [4]. In light of these expenditures, any efforts to reduce development cycle time or cost must include the testing process. Our research is addressing this concern through the development of system dynamics models of testing processes. These models enable the testing processes to be better understood by clearly identifying testing activities, their dependencies and the feedback between various testing tasks. Through the addition of metric data, the process models can be executed to reveal areas for increasing the cost effectiveness of the testing activities. The models also provide an experimental vehicle for performing various "what if" type scenarios and assessing their impact on testing cost effectiveness. For example, a system dynamics model of the testing process can reveal the differences in cycle time of projects with minimal unit testing, unit testing which detects 70% of the defects and unit testing which detects 100% of the defects. Overview of System Dynamics Modeling System dynamics modeling was developed in the late 1950’s at M.I.T. It has recently been used to model "high-level" process improvements corresponding to SEI levels of maturity [5, 3]. System dynamics models differ from traditional cost estimation models, such as COCOMO [2], in that they are not based upon statistical correlations, but rather cause-effect relationships that are observable in a real system [1]. An example of a cause-effect relationship would be a project behind schedule (cause) leading to hiring more people (effect). These cause-effect relationships constantly interact while the model is being executed, thus the dynamic interactions of the system are being modeled, hence its name. A system dynamics model can contain relationships between people, product, and process in a software development organization. The most powerful feature of system dynamics modeling is realized when multiple cause-effect relationships are connected forming a circular relationship, known as a feedback loop. The concept of a feedback loop reveals that any actor in a system will eventually be affected by its own action. A simple example of the ideas of cause-effect relationships and feedback loops affecting people, product and process can be illustrated by the following scenario: Consider the situation in which developers, perceiving their product is behind schedule (cause), modify their development process by performing less thorough unit testing (effect/cause), leading to a product of lower quality (effect/cause), but giving the temporary perception that the product is back on schedule (effect/close feedback loop). The developers perceived the product to be behind schedule, took action, and finally perceived the product to be back on schedule, as a result of their actions. Secondary effects due to the developers’ actions, however, such as the lower quality of the product, will also eventually impact the perception the developers have as to being on schedule and will necessitate