SOFTWARE TESTING, VERIFICATION AND RELIABILITY Softw. Test. Verif. Reliab. 2014; 24:386–411 Published online 21 April 2014 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/stvr.1527 Automatic test case evolution Mehdi Mirzaaghaei 1, * ,† , Fabrizio Pastore 2 and Mauro Pezzè 1,2 1 Faculty of Informatics, University of Lugano, Switzerland 2 Department of Informatics Systems and Communications, University of Milano–Bicocca, Milano, Italy SUMMARY Software systems evolve incrementally both during and after development, and many test cases become obsolete while software evolves. Updating test suites in the context of software evolution is a complex and time consuming activity. This article focuses on the problem of updating test suites automatically, and identifies eight scenarios that allow either to repair test cases or to use test cases to generate new ones, and proposes eight test evolution algorithms that automatically repair and generate test cases by adapting existing ones. This article presents a framework, TestCareAssistant (TCA), that implements the algorithms to support the evolution of test suites written in Java. The framework has been extensively evaluated on five different open source projects where it has been applied to repair 138 broken test cases, and to generate the test cases for 727 new classes and 2462 new methods. The results obtained with TCA indicate that the approach can successfully repair 90% of the broken test cases, create test cases that cover a large amount of code and complement the test cases that can be generated by state of the art techniques. Copyright © 2014 John Wiley & Sons, Ltd. Received 30 October 2012; Revised 20 February 2014; Accepted 4 March 2014 KEY WORDS: software testing; test case evolution; test case maintenance 1. INTRODUCTION Testing first and continuous integration are two key postulates of modern development processes. When adopting approaches like Scrum [1] or XP [2], software developers first design test cases, and then incrementally implement software units, and integrate them into subsystems. At each increment, developers execute the test cases and develop new ones for the added or modified func- tionality. The process of adapting and extending software systems and test cases continues after the software is released, to fix field problems and meet new customer requirements [3]. While software evolves, test suites must evolve as well. Changes in the code may invalidate test cases, while adding new functionalities requires additional test cases. For example, when developers working on programs written in Java modify some methods, they update the Junit test cases that become obsolete, and when they develop new methods and classes, they design new JUnit test cases. Updating test cases that do not compile and creating new ones are effort demanding and time consuming activities [4], and often, developers discard obsolete test cases even if easily fixable, and do not design enough test cases for new methods and classes. Techniques that automatically generate test cases can dramatically reduce the costs of generating and repairing test cases. The most popular automated approaches are either model or code based. Model based techniques use abstract models of either the software behaviour or its environment to generate test cases [5], while code based approaches generate test cases from the software source *Correspondence to: Mehdi Mirzaaghaei, Via G. Buffi 13, Faculty of Informatics, University of Lugano, CH-6904, Switzerland. † E-mail: mehdi.mirzaaghaei@gmail.com Copyright © 2014 John Wiley & Sons, Ltd.