SOFTWARE TESTING, VERIFICATION AND RELIABILITY
Softw. Test. Verif. Reliab. (2014)
Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/stvr.1542
MuRanker: a mutant ranking tool
Akbar Siami Namin
*
,†
, Xiaozhen Xue, Omar Rosas and Pankaj Sharma
Computer Science Department, Texas Tech University, Lubbock, TX, USA
SUMMARY
Mutation testing is a fault-based software testing technique in which a large number of mutants are generated
in order to assess the adequacy of test cases devised. One of the daunting problems in this area consists in
determining whether a mutant can be killed by a test case or it cannot be killed easily because the mutant
is semantically equivalent to the original programme. A possible solution, as proposed in this paper, is to
measure the complexity of each mutant and prioritize them according to how easy or hard they are to be
exposed. As a result, using a proper metric based on the mutants’ complexity, the tester may decide whether
to focus on killing easy or hard mutants first.
This paper introduces MuRanker, a mutation ranking tool that ranks mutants according to their predicted
difficulty and complexity in being detected. The tool uses distance functions to decide the difficulty level
of mutants. The computation of the distance function score is based on three representations, namely the
control-flow-graph representation, the Jimple representation and the information captured through the code
coverage data to differentiate the changes in the original and mutant programs. The tool generates a report
that displays mutants in an ordered list based on their complexity. Using this tool, the tester can choose
mutants and then devise appropriate test cases with the goal of killing them. The mutation ranking idea and
the developed tool allow the software tester to save substantial time and effort on a task that, otherwise,
would require the user to manually identify the difficult mutants. To our knowledge, MuRanker is the first
mutation tool that proposes ranking mutants to facilitate mutation testing. Copyright © 2014 John Wiley &
Sons, Ltd.
Received 3 September 2012; Revised 25 May 2014; Accepted 14 June 2014
KEY WORDS: software testing; mutation testing; testing tools; distance functions
1. INTRODUCTION
Mutation testing is the process of injecting faults into a programme. The act of inserting faults is
referred to as mutating, and the programme obtained by mutating the original programme is known
as a mutant. Mutation testing is based on the assumption that a programme is developed by a com-
petent programmer [1] and is close to being correct, that is, all of the simple faults have been
detected and removed. The competent programmer hypothesis states that a competent programmer
may develop an incorrect programme, but the incorrect programme only needs a few simple correc-
tions to be correct [2]. Mutation testing simulates the common mistakes programmers make during
the software development process [3].
This fault-based testing technique is an iterative process in which developers insert faults into a
programme to measure the effectiveness of a set of test cases and their ability to expose syntactic
changes that can resemble faults. The test case adequacy is then measured using a mutation score, the
ratio of the number of killed mutants over the total number of semantically non-equivalent mutants.
*Correspondence to: Akbar Siami Namin, Computer Science Department, Texas Tech University, Lubbock, TX, USA.
†
E-mail: akbar.namin@ttu.edu
‡
The tool and its tutorial, and a video representing the tool can be downloaded through the tool Web page:
http://www.depts.ttu.edu/cs/research/avesta/
Copyright © 2014 John Wiley & Sons, Ltd.