Model-Based Mutation Testing of Firewalls Tugkan Tuglular Department of Computer Engineering, Izmir Institute of Technology, Turkey tugkantuglular@iyte.edu.tr Fevzi Belli Department of Computer Science, Electrical Engineering and Mathematics, University of Paderborn, Germany belli@upb.de Abstract We propose a combined approach for test case generation to uncover errors both in firewall software and in its configuration. A case study validates the approach. 1. Introduction As being most important security defense of a network, firewalls have to be tested to validate that they work as specified. Most of the work known from security literature focuses on testing of firewall rules where firewall implementation is assumed error-free. However, a firewall can be hacked and programmed to behave differently from its specification. In that case, the hacked firewall is a mutant of its original. The firewall specification is mainly composed of intended security policy and allowed network protocols, which are the main focus of a hacker, or an attacker. The intended security policy consists of firewall rules which configure the firewall behavior and the allowed network protocols constitute the important part of firewall’s internal working which can be described as packet capture, decision making on the packet under consideration, and packet release. Decision making operation is carried out with respect to firewall policy and network protocols. The security policy is external to the firewall like a configuration file, whereas packet checking with respect to network protocols is implemented in the firewall software. Since the firewall policy is considered as a specification, equivalence class testing is suggested. The network protocol checking can be represented by a model. Therefore, we use here model-based mutation testing. This is the novelty of the approach. In the next section, our approach to firewall testing is explained. Section 3 presents a case study; Section 4 concludes with a brief summary and future work. 2. Approach In the equivalence class testing part of the approach, sequence of firewall rules is converted to a firewall policy tree as described in [1], which is then used as a test tree. Each node in the policy tree represents a field of a rule. A firewall rule is abstracted as “IF (<protocol>, <src_ip>, <src_port>, <dst_ip>, <dst_port>) THEN <action>”, where protocol is a network protocol, such as TCP or UDP, and action is either ALLOW or DENY. The root node of a policy tree represents the protocol field, and the leaf nodes represent the action field, intermediate nodes represent other fields in order. Every tree path starting at the root and ending at a leaf represents a rule in the policy and vice versa. The equivalence class partitioning divides the input domain of the software under test (SUT) into a finite number of partitions or equivalence classes. The equivalence classes are identified by taking each input condition – in our approach each field of the rule or each node of the policy tree – and partitioning it into two classes [2]. When generating test cases, values that a hacker might choose are considered in addition to the boundary values in equivalence classes. Assuming that the given finite state automata (FSA) correctly specifies the expected, desirable behavior of SUT, manipulation of either the state transitions or the states can be used to generate mutants of the system, i.e., to specify erroneous, undesirable situations [3]. Using this model-based approach, a mutant can be generated and it will behave in a way that the system is not supposed to behave. Basically, we can generate mutants of an FSA by inserting an extra state transition in any direction, omitting an existing arc, inserting an extra state, or omitting an existing state. Although firewalls are implemented as software, their method of input and output is network I/O. Therefore, network packets should be produced,