Applying Particle Swarm Optimization to Pairwise Testing ∗ Xiang Chen, Qing Gu † , Jingxian Qi, Daoxu Chen State Key Laboratory of Novel Software Technology Department of Computer Science and Technology, Nanjing University Nanjing, Jiangsu, China, 210093 chenxiang@dislab.nju.edu.cn Abstract—Combinatorial testing (also called interaction test- ing) is an effective specification-based test input generation technique. By now most of research work in combinatorial testing aims to propose novel approaches trying to generate test suites with minimum size that still cover all the pairwise, triple, or n-way combinations of factors. Since the difficulty of solving this problem is demonstrated to be NP-hard, existing approaches have been designed to generate optimal or near optimal combinatorial test suites in polynomial time. In this paper, we try to apply particle swarm optimization (PSO), a kind of meta-heuristic search technique, to pairwise testing (i.e. a special case of combinatorial testing aiming to cover all the pairwise combinations). To systematically build pairwise test suites, we propose two different PSO based algorithms. One algorithm is based on one-test-at-a-time strategy and the other is based on IPO-like strategy. In these two different algorithms, we use PSO to complete the construction of a single test. To successfully apply PSO to cover more uncovered pairwise combinations in this construction process, we provide a detailed description on how to formulate the search space, define the fitness function and set some heuristic settings. To verify the effectiveness of our approach, we implement these algorithms and choose some typical inputs. In our empirical study, we analyze the impact factors of our approach and compare our approach to other well-known approaches. Final empirical results show the effectiveness and efficiency of our approach. Keywords-software testing; pairwise testing; meta-heuristic search techniques; particle swarm optimization; I. I NTRODUCTION Software testing and debugging is an essential part in software development process but this process is confirmed to be laber-intensive and expensive [1]. In research and industry communities, researchers and practitioners focus on finding automatic cost-effective software testing and debugging tech- niques while still maintaining high fault detection and local- ization ability to ensure releasing high-quality software pro- ductions. Nowadays software testing research mainly concerns such issues as test coverage criterion design, test generation problem, test oracle problem, regression testing problem and * This work is supported in part by the National High-Tech Research and Development Plan of China (863) under Grant No. 2006AA01Z177, the 973 Program of China under Grant No. 2009CB320705, and the NSFC Project under Grant No. 60873027. † Correspondence author. E-mail:guq@nju.edu.cn. fault localization problem. Among these issues, test generation problem is deemed to be an important issue in software testing research. In numerous test generation techniques, com- binatorial testing is considered to be an important test input generation technique and it is an important complement to other black box testing techniques such as boundary value analysis and equivalence class partition. Combinatorial testing generates test suites according to artifacts under test. These artifacts come from either functional requirements or detailed development specifications. Given an input described by factors and levels after analyzing a specific artifact, to make an exhaustive test, we should generate a set of tests to cover every possible combination of all those factors. However the size of generated tests may be too large and the required test cost is prohibitive. To overcome this shortcoming, Cohen et al. firstly introduced the concept of combinatorial testing and proposed a greedy algorithm AETG [6]. This testing technique can extremely reduce the number of tests by only concerning combinations of a few factors and they demonstrated that the size of the optimal test suite grows at most logarithmically with the number of factors [6]. When we only want to cover all the combinations of arbitrary two factors, we call it pairwise testing. However, the reduction of tests does not seriously weaken the effectiveness of fault detection ability. Empirical studies found that a significant number of faults are triggered by the combinations of a small number of factors in a variety of application domains [11][15]. For example, Kuhn et al. analyzed error reports of a large distributed system after applying combinatorial testing and found that more than 70% bugs were found by pairwise testing. To find all the faults, they only need 6-way combinatorial testing at most [15]. Nowadays, most of research work in combinatorial testing aims to generate combinatorial test suites with minimum size. However, the difficulty of solving this problem is demonstrated to be NP-hard [10]. By now many approaches have been proposed and many tools have been developed to find opti- mal or near optimal combinatorial test suites in polynomial time [5][6][8][10][16][20][21][23][25]. Existing approaches and tools can be briefly divided into three categories: algebraic construction, greedy algorithm and meta-heuristic search. The brief description of these approaches can be found in section V. Combinatorial testing strategy has been successfully ap- 2010 IEEE 34th Annual Computer Software and Applications Conference 0730-3157/10 $26.00 © 2010 IEEE DOI 10.1109/COMPSAC.2010.17 107 2010 34th Annual IEEE Computer Software and Applications Conference 0730-3157/10 $26.00 © 2010 IEEE DOI 10.1109/COMPSAC.2010.17 107