1. Intrusion Detection Systems (I) CS 6262 Fall 02

2. Definitions Intrusion Intrusion A set of actions aimed to compromise the security goals, namely A set of actions aimed to compromise the security goals, namely Integrity, confidentiality, or availability, of a computing and networking resource Integrity, confidentiality, or availability, of a computing and networking resource Intrusion detection Intrusion detection The process of identifying and responding to intrusion activities The process of identifying and responding to intrusion activities

3. Prevent Why Is Intrusion Detection Necessary? Detect React/ Survive Security principles: layered mechanisms

4. Elements of Intrusion Detection Primary assumptions: Primary assumptions: System activities are observable System activities are observable Normal and intrusive activities have distinct evidence Normal and intrusive activities have distinct evidence Components of intrusion detection systems: Components of intrusion detection systems: From an algorithmic perspective: From an algorithmic perspective: Features - capture intrusion evidences Features - capture intrusion evidences Models - piece evidences together Models - piece evidences together From a system architecture perspective: From a system architecture perspective: Audit data processor, knowledge base, decision engine, alarm generation and responses Audit data processor, knowledge base, decision engine, alarm generation and responses

5. Components of Intrusion Detection System Audit Data Preprocessor Audit Records Activity Data Detection Models Detection Engine Alarms Decision Table Decision Engine Action/Report system activities are observable normal and intrusive activities have distinct evidence

6. Intrusion Detection Approaches Modeling Modeling Features: evidences extracted from audit data Features: evidences extracted from audit data Analysis approach: piecing the evidences together Analysis approach: piecing the evidences together Misuse detection (a.k.a. signature-based) Misuse detection (a.k.a. signature-based) Anomaly detection (a.k.a. statistical-based) Anomaly detection (a.k.a. statistical-based) Deployment: Network-based or Host-based Deployment: Network-based or Host-based Development and maintenance Development and maintenance Hand-coding of expert knowledge Hand-coding of expert knowledge Learning based on audit data Learning based on audit data

7. Misuse Detection Intrusion Patterns activities pattern matching intrusion Cant detect new attacks Example: if (src_ip == dst_ip) then land attack

8. Anomaly Detection activity measures probable intrusion Relatively high false positive rate - anomalies can just be new normal activities.

9. tcpdump BSM Network Packets Operating System Events Monitoring Networks and Hosts

10. Key Performance Metrics Algorithm Algorithm Alarm: A; Intrusion: I Alarm: A; Intrusion: I Detection (true alarm) rate: P(A|I) Detection (true alarm) rate: P(A|I) False negative rate P(¬A|I) False negative rate P(¬A|I) False alarm rate: P(A|¬I) False alarm rate: P(A|¬I) True negative rate P(¬A|¬I) True negative rate P(¬A|¬I) Bayesian detection rate: P(I|A) Bayesian detection rate: P(I|A) Architecture Architecture Scalable Scalable Resilient to attacks Resilient to attacks

11. Bayesian Detection Rate Base-rate fallacy Base-rate fallacy Even if false alarm rate P(A|¬I) is very low, Bayesian detection rate P(I|A) is still low if base-rate P(I) is low Even if false alarm rate P(A|¬I) is very low, Bayesian detection rate P(I|A) is still low if base-rate P(I) is low E.g. if P(A|I) = 1, P(A|¬I) = 10 -5, P(I) = 2×10 -5, P(I|A) = 66% E.g. if P(A|I) = 1, P(A|¬I) = 10 -5, P(I) = 2×10 -5, P(I|A) = 66% Implications to IDS Implications to IDS Design algorithms to reduce false alarm rate Design algorithms to reduce false alarm rate Deploy IDS to appropriate point/layer with sufficiently high base rate Deploy IDS to appropriate point/layer with sufficiently high base rate

12. Example ROC Curve Ideal system should have 100% detection rate with 0% false alarm Ideal system should have 100% detection rate with 0% false alarm % Detect % False Alarm IDS1 IDS2

13. Host-Based IDSs Using OS auditing mechanisms Using OS auditing mechanisms E.G., BSM on Solaris: logs all direct or indirect events generated by a user E.G., BSM on Solaris: logs all direct or indirect events generated by a user strace for system calls made by a program strace for system calls made by a program Monitoring user activities Monitoring user activities E.G., Analyze shell commands E.G., Analyze shell commands Monitoring executions of system programs Monitoring executions of system programs E.G., Analyze system calls made by sendmail E.G., Analyze system calls made by sendmail

14. Network IDSs Deploying sensors at strategic locations Deploying sensors at strategic locations E.G., Packet sniffing via tcpdump at routers E.G., Packet sniffing via tcpdump at routers Inspecting network traffic Inspecting network traffic Watch for violations of protocols and unusual connection patterns Watch for violations of protocols and unusual connection patterns Monitoring user activities Monitoring user activities Look into the data portions of the packets for malicious command sequences Look into the data portions of the packets for malicious command sequences May be easily defeated by encryption May be easily defeated by encryption Data portions and some header information can be encrypted Data portions and some header information can be encrypted Other problems … Other problems …

15. Architecture of Network IDS Network libpcap Event Engine Policy Script Interpreter Packet stream Filtered packet stream Event stream Alerts/notifications Policy script Event control tcpdump filters

16. Firewall Versus Network IDS Firewall Firewall Active filtering Active filtering Fail-close Fail-close Network IDS Network IDS Passive monitoring Passive monitoring Fail-open Fail-open FW IDS

17. Requirements of Network IDS High-speed, large volume monitoring High-speed, large volume monitoring No packet filter drops No packet filter drops Real-time notification Real-time notification Mechanism separate from policy Mechanism separate from policy Extensible Extensible Broad detection coverage Broad detection coverage Economy in resource usage Economy in resource usage Resilience to stress Resilience to stress Resilience to attacks upon the IDS itself! Resilience to attacks upon the IDS itself!

18. Eluding Network IDS What the IDS sees may not be what the end system gets. What the IDS sees may not be what the end system gets. Insertion and evasion attacks. Insertion and evasion attacks. IDS needs to perform full reassembly of packets. IDS needs to perform full reassembly of packets. But there are still ambiguities in protocols and operating systems: But there are still ambiguities in protocols and operating systems: E.G. TTL, fragments. E.G. TTL, fragments. Need to normalize the packets. Need to normalize the packets.

19. Insertion Attack A T X T A C A T T A CKK T X T C A A K End-System sees: IDS sees: Attackers data stream Examples: bad checksum, TTL.

20. Evasion Attack A T T C K A T T A CKT T C A A K End-System sees: IDS sees: Attackers data stream Example: fragmentation overlap

21. DoS Attacks on Network IDS Resource exhaustion Resource exhaustion CPU resources CPU resources Memory Memory Network bandwidth Network bandwidth Abusing reactive IDS Abusing reactive IDS False positives False positives Nuisance attacks or error packets/connections Nuisance attacks or error packets/connections