Detection and Resolution of Anomalies in Firewall Policy Rules Muhammad Abedin, Syeda Nessa, Latifur Khan, Bhavani Thuraisingham Department of Computer Science, The University of Texas at Dallas.

Outline Firewall Concepts and Definitions The Problem Existing Works Our Contribution Anomaly Resolution Algorithms Correctness Cost Analysis Example Merging Rules Conclusion and Future Directions

Firewalls Firewall: System acting as an interface of a network to one or more external networks. Implements the security policy of the network By deciding which packets to let through Based on rules defined by the network administrator.

Firewall Rules Firewall Rules: Mostly Custom-designed and Hand-written. Must be defined and maintained carefully. Result of any slight mistake in defining the rules: Allow unwanted traffic to be able to enter or leave the network Deny passage to quite legitimate traffic. Manual Definition and Maintenance: Complex, Error-prone, Costly, Inefficient As number of rules increase Becomes virtually unmanageable.

Our Research Focuses on: Automating the maintenance of Firewall Rule Set. Making the rule set error-free detecting the conflicts and anomalies in the rules Resolving the errors found Increasing the efficiency Reducing the number of rules in the set.

Existing Work There has been research in firewall policy analysis. Static Analysis and Detection of Anomalies Most of the current work is based on only detection of anomalies. Some work on Resolution of Anomalies

Our Contribution This paper presents: Complete set of definitions of relations between rules and possible anomalies. A set of algorithms to simultaneously detect and resolve anomalies in a given rule set. Proof of correctness of the algorithms and cost analysis. An algorithm to merge rules whenever possible to reduce the number of rules.

Representation of Rules A rule contains Set of criteria Direction, Protocol, Source IP, Source Port, Destination IP, Destination Port. Action performed on a packet matching the criteria ACCEPT or REJECT. A complete rule is defined by the ordered tuple <Dir, Proto, Src IP, Src Port, Dest IP, Dest Port, Action> Example: <OUT, TCP, 192.168.20.*, ANY, 64.233.179.104, 80, ACCEPT>

Relations between Rules Attributes of a rule: Defined as a range of values Rule Can be compared using set relations. Relation between two rules can be Disjoint Exactly Matching Inclusively Matching Correlated

Disjoint Rules Two rules r and s are disjoint, denoted as , if they have at least one criterion for which they have completely disjoint values. Example: <IN, TCP, 64.233.179.104, 80, 192.168.20.*, ANY, ACCEPT> <IN, TCP, 64.233.179.104, 80, 172.16.20.*, ANY, REJECT>

Exactly Matching Two rules r and s are exactly matched, denoted by , if each criterion of the rules match exactly. Example: <IN, TCP, 64.233.179.104, 80, 192.168.20.*, ANY, ACCEPT>

Inclusively Matching A rule r is a subset, or inclusively matched of another rule s, denoted by , if there exists at least one criterion for which r’s value is a subset of s’s value and for the rest of the attributes r’s value is equal to s’s value. Example: <IN, TCP, 64.233.179.104, 80, 192.168.20.3, ANY, ACCEPT> <IN, TCP, 64.233.179.104, ANY, 192.168.20.*, ANY, ACCEPT>

Correlated Two rules r and s are correlated, denoted by , if r and s are not disjoint, but neither is the subset of the other. Example: <IN, TCP, 64.233.179.104, ANY, 192.168.20.3, ANY, ACCEPT> <IN, TCP, 64.233.179.104, 80, 192.168.20.*, ANY, REJECT>

Possible Anomalies Between Two Rules Definition of anomalies in terms of relations: Shadowing Anomaly Rule r is shadowed by rule s if s precedes r in the policy and s can match all the packets matched by r. Example: r = <IN, TCP, 64.233.179.104, ANY, 192.168.20.*, ANY, ACCEPT> s = <IN, TCP, 64.233.179.104, 80, 192.168.20.3, ANY, ACCEPT> Correlation Anomaly Rules r and s are correlated if they have different actions and r matches some packets that match s and s matches some packets that r matches. r = <IN, TCP, 64.233.179.104, ANY, 192.168.20.3, ANY, ACCEPT> s = <IN, TCP, 64.233.179.104, 80, 172.16.20.*, ANY, REJECT>

Possible Anomalies Between Two Rules (Contd.) Definition of anomalies in terms of relations (continued): Redundancy Anomaly A redundant rule r performs the same action on the same packets as another rule s such that if r is removed the security policy will not be affected. Example: r = <IN, TCP, 64.233.179.104, 80, 192.168.20.3, ANY, ACCEPT> s = <IN, TCP, 64.233.179.104, ANY, 192.168.20.*, ANY, ACCEPT>

Anomaly Resolution Algorithms We resolve the anomalies as follows: Shadowing anomaly: Exactly matched keep the one with the REJECT action. Inclusively matched Reorder the rules to bring the subset rule before the superset rule. Correlation anomaly: Break down the rules into disjoint parts and insert them into the list. Of the part that is common to the correlated rules, keep the one with the REJECT action. Redundancy anomaly: Remove the redundant rule.

Approach Two global lists of firewall rules Incremental approach: old_rules_list Contains the rules as they are in the original firewall configuration new_rules_list Will contain the output of the algorithm, a set of firewall rules without any anomaly. Incremental approach: Take each rule in the old_rules_list Insert it into new_rules_list in such a way that new_rules_list always remains free from anomalies.

The Algorithms We present the following algorithms to resolve the anomalies RESOLVE-ANOMALIES Processes the old_rules_list to produce the anomaly-free new_rules_list. INSERT Inserts a rule into the new_rules_list in such a way that the list remains anomaly free. RESOLVE Detects and resolves anomalies between two given non-disjoint rules. SPLIT Splits two non-disjoint rules into disjoint parts for a given attribute.

Interaction of the Algorithms RESOLVE- ANOMALIES Inserts each rule into new_rules_list INSERT old_rules_list Firewall Rules Inserts disjoint parts into new_rules_list Resolves non-disjoint pairs of rules Splits correlated pairs of rules SPLIT RESOLVE

Illustrative Example Initialization Step-1 old_rules_list <IN, TCP, 129.110.96.117, ANY, 129.110.96.*, 80, REJECT> <IN, TCP, 129.110.96.*, ANY, 129.110.96.*, 80, ACCEPT> <IN, TCP, 129.110.96.*, ANY, 129.110.96.80, 80, ACCEPT> new_rules_list Empty Step-1 Rule-1 is inserted into empty new_rules_list

Illustrative Example (Contd) Step-2 Rule-2 is superset of rule-1 Inserted after rule-1 in new_rules_list new_rules_list: <IN, TCP, 129.110.96.117, ANY, 129.110.96.*, 80, REJECT> <IN, TCP, 129.110.96.*, ANY, 129.110.96.*, 80, ACCEPT>

Illustrative Example (Contd) Step-3 Rule-3 is correlated with rule-1 of new_rules_list These rules will be split into disjoint parts by algorithm SPLIT The disjoint and common parts will be inserted by algorithm INSERT

Illustrative Example (Contd) Rule-3: Original: <IN, TCP, 129.110.96.*, ANY, 129.110.96.80, 80, ACCEPT> After Split: <IN, TCP, 129.110.96.117, ANY, 129.110.96.80, 80, ACCEPT> <IN, TCP, 129.110.96.1-116, ANY, 129.110.96.80, 80, ACCEPT> <IN, TCP, 129.110.96.118-254, ANY, 129.110.96.80, 80, ACCEPT> Rule-1: <IN, TCP, 129.110.96.117, ANY, 129.110.96.*, 80, REJECT> After Split <IN, TCP, 129.110.96.117, ANY, 129.110.96.80, 80, REJECT > <IN, TCP, 129.110.96.117, ANY, 129.110.96.1-79, 80, REJECT> <IN, TCP, 129.110.96.117, ANY, 129.110.96.81-254, 80, REJECT>

Illustrative Example (Contd) We insert these split rules into new_rules_list and get the final list: <IN, TCP, 129.110.96.1-116, ANY, 129.110.96.80, 80, ACCEPT> <IN, TCP, 129.110.96.118-254, ANY, 129.110.96.80, 80, ACCEPT> <IN, TCP, 129.110.96.117, ANY, 129.110.96.1-79, 80, REJECT> <IN, TCP, 129.110.96.117, ANY, 129.110.96.81-254, 80, REJECT> <IN, TCP, 129.110.96.117, ANY, 129.110.96.80, 80, REJECT> <IN, TCP, 129.110.96.*, ANY, 129.110.96.*, 80, ACCEPT>

Cost Analysis of INSERT The cost of running the algorithm depends on the nature of the rules in the input. A disjoint or superset rule is inserted into the list without further call to INSERT. In this case INSERT must check the whole new_rules_list. A subset rule is inserted just before the superset rule. In the worst case INSERT may check the whole new_rules_list. A rule equal to any rule in the new_rules_list is discarded. In the worst case the whole new rules list may have to be traversed. Two correlated rules will be divided into a set of mutually disjoint rules. In the worst case the number of rules thus generated will be up to twice the number of attributes plus one, and these rules will be inserted into the new rules list by invoking INSERT recursively.

Cost Analysis of RESOLVE-ANOMALIES Invokes INSERT once for each rule in new_rules_list in a for loop. Removes the redundancy anomalies in a second for loop The running time of Resolve-Anomalies is dominated by the number of times INSERT is invoked recursively. This depends on the number of correlated rules in the input. Worst case: If all rules are correlated, the running time may deteriorate to exponential order.

Merging Rules Given an anomaly-free list we can merge the rules with consecutive attribute values to reduce the number of rules. We can construct a tree representing the anomaly-free rules. Nodes will represent attributes. Edges will represent value ranges of the attributes. To find rules with consecutive attribute values and safe to merge Traverse the tree in post-order

Merge Algorithms TREEINSERT MERGE Takes as input a rule and a node of the tree Constructs a tree from the input rules MERGE Traverses the tree in post-order Invokes MERGE on the children of the node Checks pairs of edges if Values of ranges are consecutive Sub-trees of the edges match exactly. Matching edge pairs are merged into single edges

Illustrative Example of the Merge Algorithm To illustrate the merge operation, consider the following rule set:

Generated Tree From this rules list we generate a tree by the TREEINSERT Algorithm, on which MERGE is applied. Edges out of node-9 can be merged into one edge.

Application of MERGE In this figure, we show an intermediate state of the tree: node 7’s children are going to be merged next. After MERGE is complete on the tree, the only rule is <IN, TCP, 202.80.169.29-110, 483-484, 129.110.96.64-164, 100-127,ACCEPT>.

Conclusion and Future Works Our work: Presents an automated process for detecting and resolving such anomalies. Establishes the complete definition and analysis of the relations between rules. In future: This analysis can be extended to distributed firewalls Data mining techniques can be used to analyze the log files of the firewall and discover other kinds of anomalies after the rules have been made free from anomaly by applying the algorithms in this paper.

Thank You