Alexandros Savvopoulos csdp1107 Performance improvement of intrusion detection with fusion of multiple sensors Vrushank Shah, Akshai K. Aggarwal, Nirbhay Chaubey HY562 Advanced Topics in Databases Alexandros Savvopoulos csdp1107

Overview General Information about IDS Definitions IDS Categories Data Collection in IDS Approaches for evaluation intrusions Assigning scores in IDS Problem & Solution in IDS Distributed IDS approach Alert-to-mass conversion The reliability of intrusion detection systems Proposed Fusion Rule Dempster-Shafer theory Dempster-Shafer rule 2

Overview KDD99 Dataset Experimental setup Results Comparison References Conclusions 3

Definitions What is intrusion? Any set of actions which violates the security protocol of a computer network system What is IDS? Intrusion Detection System It monitors network traffic and attempts to identify unusual or suspicious activity It raises an alarm for any abnormal behaviors What is a distributed IDS system? It is a framework Multiple heterogeneous IDS systems are deployed to sniff the incoming network traffic 4

IDS Categories Active IDS It is also known as Intrusion Detection and Prevention System (IDPS) It is configured to automatically block suspected attacks without any intervention required by an operator The main advantage is that provide real-time corrective action in response to an attack Passive IDS It configures to only monitor and analyze network traffic activity It alerts an operator to potential vulnerabilities and attacks It is not capable of performing any protective or corrective functions on its own 5

IDS Categories NIDS Network Intrusion Detection System It consists from a network appliance (or sensor) with a Network Interface Card (NIC) operating in promiscuous mode a separate management interface It is placed along a network segment or boundary and monitors all traffic on that segment HIDS Host Intrusion Detection System HIDS and software applications (agents) installed on work stations which are to be monitored The agents monitor the operating system and write data to log files and/or trigger alarms HIDS can only monitor the individual work stations on which the agents are installed It can’t monitor the entire network 6

IDS Categories Signature-based IDS It is also known as Knowledge-based IDS Each intrusion leaves a footprint behind Examples nature of data packets failed logins The footprints are called signatures They can use signatures in order to identify and prevent the same attacks in the future. Anomaly-based IDS It is also known as Behavior-based IDS It references a baseline or learned pattern of normal system activity to identify active intrusion attempts Deviations from this baseline or pattern cause an alarm to be triggered 7

Data Collection in IDS Useful data from IDS source and destination IP addresses ports packet headers network traffic statistics Examples of IDS Tcpdump command line tool Wireshark open source packet sniffer Snort open source IDS packet capture and signature matching Suricata is a free and open source, fast and robust network threat detection engine Packet Header Anomaly Detector (PHAD) that learns the normal range of values for 33 fields of the Ethernet, IP, TCP, UDP, and ICMP protocols Network Anomaly Detector (NETAD) 8

Approaches for evaluation intrusions https://www-users.cs.umn.edu/~kumar001/MINDS/papers/siam2003.pdf 9

Assigning scores in IDS Assume that for a given network traffic each connection is assigned a score value [vertical line] the real attack curve [dash line] zero for non-intrusive (normal) network connections one for intrusive connections the predicted attack curve [full line] for each connection it is equal to its assigned score https://www-users.cs.umn.edu/~kumar001/MINDS/papers/siam2003.pdf 10

Problem & Solution in IDS What is the problem with IDS? The technological advancement in computer network system Drawbacks Higher false alarm rate Lower detection rate What is the solution? The use of multiple sensors/intrusion detection systems (Distributed IDS systems) Systems which are dissimilar in nature Limit the detection performance of intrusion detection system 11

Distributed IDS approach 12

Alert-to-mass conversion In Distributed IDS, each IDS raises an alert for the presence of an attack the alerts can be positive or negative the alerts generated by IDS which are converted to a mass value using the formula proposed by Jøsang P the positive evidence in favour of hypothesis H N the negative evidence opposing the hypothesis H C the constant which is equal to 2 for binary frame of hypothesis 13

Reliability of intrusion detection system The reliability is the level of trust about the IDS evidences for detecting intrusion indicates the relative stability of IDS whose values are from 0 to 1 Some of IDS are completely reliable, but others are not The success of fusion depends on the accuracy of evidence provided by the individual IDS Majority of fusion rules all evidences to be equally reliable assign the same weight-age during the fusion process The evidence provided by individual IDS zero reliability should be completely ignored higher reliability should be given more weightage 14

Reliability of intrusion detection system One major concern in incorporating reliability of IDS into the fusion rule the problem of obtaining reliability values Two approaches for finding reliability The existence of conflict between the evidence indicates the presence of an unreliable IDS may cause the fusion result to be away from reality 2. It relates the reliability with the true alert rate of IDS In this approach they assume that the IDS having the highest true alert rate the highest weightage in fusion process the lower false alert rate higher reliability given highest weightage in fusion process 15

Dempster-Shafer theory What is the Dempster-Shafer theory? A mathematical theory that combines the evidence from multiple sources of information calculates the probability of an event Dempster–Shafer theory is a generalization of the Bayesian theory of subjective probability  In Bayesian Theory, the probability is interpreted as reasonable expectation representing a state of knowledge It is also known as evidence theory The Dempster–Shafer theory proposed by Arthur Dempster modified by Glenn Shafer the first mathematical theory which combines uncertain information of sources The fusion rule proposed under Dempster-Shafer framework is called as Dempster-Shafer rule 16

Dempster-Shafer theory The Dempster-Shafer theory is used to combine masses from n evidence sources In Dempster-Shafer framework there is the basic belief assignment (BBA) for the number of evidence sources n ≥ 2 2Θ : the set of all subsets of Θ = { θ1 , θ2 , θ3 , … , θn } Example for power set of Θ: Θ = { a , b } with the empty set 2Θ : the set of all subsets of Θ 2Θ = { ∅ , {a} , {b} , Θ} The theory of evidence assigns a belief mass to each element of the power set. Formally, a function called basic belief assignment (BBA): m : 2Θ → [0,1] BBA has two properties: 1. 2. 17

Dempster-Shafer rule Let m1(B) and m2(C): two independent masses from two sources of evidence 18

Dempster-Shafer rule The limitations of Dempster–Shafer rule does not incorporate the reliability of sources whose evidences have to be fused contains all the sources of evidence which are equally reliable in an environment consisting of many hypotheses and many sources If sources of evidences are highly conflicting If analyst blindly believes on the result Goodman said that it is difficult to decide whether to accept or reject the result of such fusion rule the DS rule completely fails the decision can be misleading 19

Proposed Fusion Rule CRF(A) the conjunctive reliability value about A DRF(A) the disjunctive reliability value about A CRF and DRF values act as weighting factors to compromise between conjunctive mass and disjunctive mass Rn the reliability value of nth source of evidence 20

Distributed IDS approach 21

KDD99 dataset KDD99 (http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html) is designed for evaluation of intrusion in computer networks includes a wide variety of intrusions simulated in a military network environment 4GB of compressed binary TCP dump data from seven weeks of network traffic the dataset is available in tcpdump format The first line of dataset: 0,tcp,http,SF,215,45076,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00, 1.00,0.00,0.00,0,0,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,normal 22

KDD99 dataset consists of 4,900,000 single connection vectors each of them contains 41 features are labeled as either normal or an attack with exactly one specific attack type each connection record consists of about 100 bytes 23

KDD99 dataset Feature name Description Type duration length(number of seconds) of the connection continuous protocol_type type of the protocol e.g. tcp, udp, etc discrete service network service on the destination, e.g. http, telnet etc src_bytes number of data bytes from source to destination dst_bytes number of data bytes from destination to source flag normal or error status of the connection land 1 if connection is from/to the same host/port; 0 otherwise wrong_fragment number of “wrong” fragments urgent number of urgent packets Table: Basic features of individual TCP connections 24

Experimental setup They used four heterogeneous intrusion detection systems Snort (https://www.snort.org/) Suricata (https://suricata-ids.org/) Packet Header Anomaly Detector (PHAD) (https://cs.fit.edu/~mmahoney/paper3.pdf) Network Anomaly Detector (NETAD) Experiment resources three third-generation Intel Core i5 processors (1.6 GHz) first machine with signature-based IDS second machine with anomaly-based detectors third machine acts as an attacker machine having KDD99 Dataset The packets of the dataset are being replayed with TCPREPLAY operating system is installed is Linux Ubuntu with 4 GB RAM Signature-based intrusion detectors Anomaly-based detectors 25

Experimental setup The experiments are focused on detection of smurf attack The frame of discernment is Θ = {smurf, −smurf, θ} In KDD99 dataset  there are total 1944 smurf attacks 26

Results TPR True Positive Rate FPR False Positive Rate PPV Positive Prediction Value NPV Negative Prediction Value 27

Results 28

Results TPR True Positive Rate FPR False Positive Rate PPV Positive Prediction Value NPV Negative Prediction Value 29

Results 30

Comparing the Results 31

Comparing the Results 32

References General Papers https://www-users.cs.umn.edu/~kumar001/MINDS/papers/siam2003.pdf https://www.researchgate.net/publication/324766936_A_Survey_on_Anomaly_Based_Host_Intrusion_Detection_System http://www.isecure-journal.com/article_66995.html https://ieeexplore.ieee.org/document/6524462 The main paper which i presented https://link.springer.com/article/10.1007/s40747-016-0033-5

Conclusions Distributed alert fusion can be achieved with Dempster–Shafer rule The proposed alert fusion system improves the performance of detection The reduction in false alerts was achieved The proposed rule incorporates variable reliability of IDS measured either from conflict between IDS or from true positive rate of IDS My conclusion: The paper was only 7 pages it had many information about free-open source dataset figures which explain the results in the most important problem increasing of TPR reducing of FPR i would prefer more information about mathematical theory 34