An Initial Investigation of Test Driven Development in Industry Boby George Department of Computer Science North Carolina State University Raleigh, NC 27695-7534 (+1) 919 601 2922 bobygeorge@ncsu.edu Laurie Williams Department of Computer Science North Carolina State University Raleigh, NC 27695-7534 (+1) 919 513 4151 williams@csc.ncsu.edu ABSTRACT Test Driven Development (TDD) is a software development practice in which unit test cases are incrementally written prior to code implementation. In our research, we ran a set of structured experiments with 24 professional pair programmers. One group developed code using TDD while the other a waterfall-like approach. Both groups developed a small Java program. We found that the TDD developers produced higher quality code, which passed 18% more functional black box test cases. However, TDD developer pairs took 16% more time for development. A moderate correlation between time spent and the resulting quality was established upon analysis. It is conjectured that the resulting high quality of code written using the TDD practice may be due to the granularity of TDD, which may encourage more frequent and tighter verification and validation. Lastly, the programmers which followed a waterfall-like process often did not write the required automated test cases after completing their code, which might be indicative of the tendency among practitioners toward inadequate testing. This observation supports that TDD has the potential of increasing the level of testing in the industry as testing as an integral part of code development. Keywords Software Engineering, Test Driven Development, Extreme Programming, Agile Methodologies, Software Experimentation. 1. INTRODUCTION Test Driven Development (TDD) [3], a software development practice used sporadically for decades [9] has gained added visibility recently as a practice of Extreme Programming (XP) [1, 2, 11, 12]. TDD is also known by names such as, Test First Design (TFD), Test First Programming (TFP) and Test Driven Design (TDD). The practice evolves the design of a system starting from the unit test cases of an object. Writing test cases and implementing that object or object methods then triggers the need for other objects/methods. An important rule in TDD is: “If you can’t write test for what you are about to code, then you shouldn’t even be thinking about coding.” [6] An object is the basic building block of Object Oriented Programming (OOP). Unless objects are designed judiciously, dependency problems, such as tight coupling of objects and fragile super classes (inadequate encapsulation) can creep in. These problems could result in a large complex code base that compiles and runs slowly. XP originator Kent Beck asserts, “Test-first code tends to be more cohesive and less coupled than code in which testing isn’t a part of the intimate coding cycle.” [4] TDD proponents argue that reduce coupling occurs because the practice guides towards the building of objects that are actually needed (to pass test cases based on the requirements) rather than building objects that are thought to be needed (due to possible improper understanding of requirements). Moreover, TDD enables continuous regression testing, which improves code quality [3]. Although intriguing, software practitioners can be concerned about the lack of upfront design in TDD and the need to make design decisions at every stage of development. This necessitates the need to empirically analyze and quantify the effectiveness of this practice. This research outlined in this paper empirically examines the following two hypotheses: 1. The TDD practice will yield code with superior external code quality when compared with code developed with a more traditional waterfall-like practice. External code quality will be assessed based on the number of functional (black-box test cases) test cases passed. 2. Programmers who practice TDD will develop code faster than developers who develop code with a more traditional waterfall- like practice. Programmers’ speed will be measured by the time to complete (hours) a specified program. To investigate these hypotheses, research data was collected from three sets of structured experiments conducted with professional developers. 2. BACKGROUND AND RELATED WORK In this section, we first describe the TDD practice in detail. Then, we describe an empirical study of TDD that has been completed by researchers in Germany. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC 2003, Melbourne, Florida USA. Copyright 2003 ACM 1-58113-624-2/03/03…$5.00.