1. CIT 485: Advanced Cybersecurity
Intrusion Detection/Prevention Systems

2. Topics Definitions and Goals Models of Intrusion Detection
False Positives
Architecture of an IDS
Example IDS: bro and snort
Active Response (IPS)
Host-based IDS and IPS
IDS Evasion Techniques
Honeypots

3. IDS Terminology Intrusion Intrusion detection Intrusion prevention
Actions aimed at compromising the security of the target (confidentiality, integrity, availability of computing/networking resources)
Intrusion detection
The identification through intrusion signatures and report of intrusion activities
Intrusion prevention
The process of both detecting intrusion activities and managing automatic responsive actions throughout the network

4. Deep Packet Inspection (DPI)
Most network devices examine layer 2-4 headers.
DPI means examining application layer and body data.
Most IDS/IPS devices perform DPI.
Web Application Firewalls also apply DPI.
Attackers attempt to bypass DPI with
Tunneling one protocol inside another protocol.
Encrypting data.

5. Goals of IDS Detect wide variety of intrusions
Previously known and unknown attacks.
Need to adapt to new attacks or changes in behavior.
Detect intrusions in timely fashion
May need to be be real-time, especially when system responds to intrusion.
Problem: analyzing commands may impact response time of system.
May suffice to report intrusion occurred a few minutes or hours ago.

6. Goals of IDS Present analysis in easy-to-understand format.
Ideally a binary indicator.
Usually more complex, allowing analyst to examine suspected attack.
User interface critical, especially when monitoring many systems .
Be accurate
Minimize false positives, false negatives.
Minimize time spent verifying attacks, looking for them.

7. Models of Intrusion Detection
Anomaly detection (statistical)
First develop profile or baseline of normal traffic.
Alert when actions depart too far from profile.
Most use statistical and machine learning algorithms.
Misuse detection (rule-based)
Create signatures based on attack profiles.
Look for signatures, hope for no new attacks.
Signatures include data from
Network headers (IP addresses, ports, protocol, etc.)
Byte strings that match content in packet body.
Specification based
Manually create profiles of normal traffic.
Alert when traffic deviates from profiles.

8. Statistical Anomaly Detection
Bayesian Algorithms
Compute probabilities using baseline traffic.
Identify improbable events as anomalies.
Association rules
Build if/then style rules from baseline traffic.
Ex: if >5 login failures + login success, then alert.
Correlational techniques
Computes correlations, which measure how likely a packet is to be associated with another packet, from baseline to identify unlikely traffic patterns.

9. AI Anomaly Detection
AI techniques like machine learning and neural networks attempt to find a decision boundary based on baseline data that divides normal and malicious traffic.

10. Additional Anomaly Algorithms
Filtering Algorithms
Filter out network traffic that closely matches baseline.
Reduces amount of data for sophisticated algorithms to process.
Fuzzy logic
Uses real numbers to handle uncertainity instead of traditional 0/1 (true/false) logic.
Can build both statistical and AI based anomaly detection models based on fuzzy logic.

11. Possible Alarm Outcomes
Intrusion Attack
No Intrusion Attack
Alarm
Sounded No
True Positive
False Positive
True Negative
False Negative

12. IDS Tuning
Configure IDS for operating systems and software used on your network.
Avoid useless alerts, such as Windows attacks on Linux network.
Reduces false positives.
IDS Tuning Process
Start with default or community rulsets.
Identify alerts that are not helpful.
Turn off rules that are false positives or useless.
Tuning is a continual activity, as your network changes and as you receive updated rules.

13. Base-Rate Fallacy
Difficult to create IDS with high true-positive rate and low false-negative rate.
If #intrusions small compared to normal traffic, then IDS will produce many false positives for each intrusion.
Effectiveness of IDS can be misinterpreted due to a statistical error known as the base-rate fallacy.
This type of error occurs when the probability of some conditional event is assessed without considering the “base rate” of that event.

14. Base-Rate Fallacy Example
Example case
IDS 99% accurate, 1% false positives or negatives
IDS generates 1,000,100 log entries.
Base rate is 100 malicious events of 1,000,100 examined.
Results
Of 100 malicious events, 99 will be detected as malicious, which means we have 1 false negative.
Of 1,000,000 benign events, 10,000 will be mistakenly identified as malicious. That is, we have 10,000 false positives!
Thus, 10,099 alarms sounded, 10,000 of which are false alarms. Roughly 99% of our alarms are false alarms.

15. IDS Components IDS Manager Untrusted Internet IDS Sensor Firewall
router
Firewall

16. IDS Architecture A modern IDS is distributed with multiple sensors.
Sensors gathers data for analysis from hosts or network.
Manager analyzes data obtained from sensors according to its internal rules.
Notifier acts on manager results.
May simply notify security officer.
May reconfigure sensors or manager to alter collection, analysis methods.
May activate response mechanism.
Hierarchical IDS divide network into clusters, with each cluster having its own manager and notifier that react in real time while forwarding data to central manager.

17. Host-Based Sensors Obtain information from logs
May use many logs as sources.
May be security-related or not.
May use virtual logs if agent is part of the kernel.
Agent generates its information
Analyzes state of system.
Treats results of analysis as log data.

18. Network-Based Sensors
Sniff traffic from network.
Use hubs, SPAN ports, or taps to see traffic.
Need sensors on all switches to see entire network.
Deep packet inspection (DPI).
Sensor needs same view of traffic as destination
Attacker may send packets with TTL set so that they arrive at destination but expire before reaching sensor.
Packet fragmentation and reassembly works differently on different OSes, so sensor sees different packet than destination in some cases.
End-to-end encryption defeats content monitoring
Not traffic analysis, though.

19. Aggregation of Information
Sensors produce information at multiple layers of abstraction.
Application-monitoring sensors provide one view of an event.
System-monitoring sensors provide a different view of an event.
Network-monitoring sensors provide yet another view (involving many packets) of an event.

20. Notifier Accepts information from manager Takes appropriate action
Generate audio or visual alert on IDS console.
Page, call, IM, or security officer.
Send syslog message or SNMP trap to record event.
Start incident response application.
Use geolocation and whois to identify attacker.
Increase monitoring to capture full PCAP traces for IPs involved in event that triggered alert.
Rate-limit contacts so a single problem does not result in an overwhelming flood of notices.

21. HIDS Example Host-based Intrusion Detection System (HIDS)
OSSEC detects intrusions based on
File checksums, to detect when files are changed.
Log file monitoring, to look for signatures of attacks.
Process monitoring, to look for unusual process activity.

22. Bro: anomaly detection IDS
Bro is a network analysis framework.
Bro collects network data.
Interprets network data based on user-created scripts.

23. Snort: Rule-based IDS
Snort is a packet sniffer that compares packets against a set of rules containing attack signatures.

24. Snort Rules Rule Header Rule Body Action: pass, log, alert
Network Protocol
Source Address (Host or Network) + Port
Destination Address (Host or Network) + Port
Rule Body
Content: packet ASCII or binary content
TCP/IP flags and options to match
Message to log, indicating nature of misuse detected

25. Snort Rule Example Example: rule for ssh shell code exploit
alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (msg:"EXPLOIT ssh CRC32 overflow NOOP"; flow:to_server,established; content:"| |"; reference:bugtraq,2347; reference:cve,CVE ; classtype:shellcode-detect; sid:1326; rev:3;)

26. IDS Deployment IDS deployment should reflect your threat model.
Major classes of attackers:
External attackers intruding from Internet.
Internal attackers intruding from your LANs.
Where should you place IDS systems?
Perimeter (outside firewall)
DMZ
Intranet
Wireless

27. IDS Deployment Need to monitor Stealthy deployment External traffic.
Local network traffic.
Wireless traffic.
Stealthy deployment
IDS should not receive but not send traffic to avoid detection by attackers.
Configure sensor w/o IP.
Can use separate network or cut wire to remove send capability.

28. Snort Web Interface

29. Sguil NSM Console

30. Intrusion Prevention Systems
What else can you do with IDS alerts?
Identify attack before it completes.
Prevent it from completing.
How to prevent attacks?
Directly: IPS drops packets, kills TCP sessions.
Indirectly: IPS modifies firewall rules.
Is IPS a good idea?
How do you deal with false positives?

31. IPS Deployment Types
Inline
Intranet
IPS
Non-Inline
IPS
Intranet

32. Active Responses by Network Layer
Data Link: Shut down a switch port. Only useful for local intrusions. Rate limit switch ports.
Network: Block a particular IP address.
Inline: can perform blocking itself.
Non-inline: send request to firewall.
Transport: Send TCP RST or ICMP messages to sender and target to tear down TCP sessions.
Application: Inline IPS can modify application data to be harmless: /bin/sh -> /ben/sh

33. Host IDS and IPS Anti-virus and anti-spyware Log monitors
AVG anti-virus, SpyBot S&D
Log monitors
swatch, logwatch
Integrity checkers
tripwire, osiris, samhain
Monitor file checksums, etc.
Application shims
mod_security (usually called a WAF)

34. Evading IDS and IPS Alter appearance to prevent sig match
URL encode parameters to avoid match.
Use ‘ or 783>412-- for SQL injection.
Alter context
Change TTL so IDS sees different packets than target hosts receives.
Fragment packets so that IDS and target host reassemble the packets differently.

35. Fragment Evasion Techniques
Flood of fragments
DoS via heavy use of CPU/RAM on IDS.
Tiny fragment
Break attack into multiple fragments, none of which match signature.
ex: frag 1:“cat /etc”, frag 2: “/shadow”
Overlapping fragments
Offset of later fragments overwrites earlier fragments.
ex: frag 1: “cat /etc/fred”, frag 2: offset=10, “shadow”
Different OSes deal differently with overlapping.

36. Web Evasion Techniques
URL encoding
GET /%63%67%69%2d%62%69%6e/bad.cgi
/./ directory insertion
GET /./cgi-bin/./bad.cgi
Long directory insertion
GET /junklongdirectorypathstuffhereuseless/../cgi-bin/bad.cgi
IDS may only read first part of URL for speed.
Tab separation
GET<tab>/cgi-bin/bad.cgi
Tabs usually work on servers, but may not be in sig.
Case sensitivity
GET /CGI-BIN/bad.cgi
Windows filenames are case insensitive, but signature may not be.

37. Countering Evasion Keep IDS/IPS signatures up to date.
On daily or weekly basis.
Use both host and network IDS/IPS.
Host-based harder to evade as runs on host.
Fragment attacks can’t evade host IDS.
Network IDS still useful as overall monitor.
Tune IDS/IPS to handle based on experience
False positives
False negatives

38. Honeypots
Honeypot: a system designed solely for intruders to attack in order to accomplish one or more of the following goals. Multiple honeypots or a single honeypot on multiple IPs can form a honeynet.
Detect intrusions with very few false positives, since legitimate users have no reason to access honeypot.
Monitor attacker activities to determine targeted assets, origin, motivation, capabilities, etc.
Waste intruder time attacking honeypot, so that defender has time to respond to incident.

39. Low Interaction Honeypots
honeyd: responds to probes on a set of unused IP addresses via shell scripts that can return banners for simple scans like nmap –sV. nepenthes: emulates vulnerable Windows services to collect exploits and malware. Dionaea: scriptable honeypot designed to be able to emulate wide variety of vulnerable services to collect exploits and malware. Fakenet: simulates DNS, HTTP, HTTPS to dynamically analyze malware. Returns reasonable responses to malware requests.

40. Medium Interaction Honeypots
Kippo is a medium interaction ssh honeypot designed to log brute force attacks and attacker shell commands. Inspired by Kojoney, a LI ssh honeypot.
Fake filesystem with the ability to add/remove files.
Possibility of adding fake file contents so the attacker can 'cat' files such as /etc/passwd. Only minimal file contents are included.
Saves files downloaded with wget for later inspection
Deception: ssh pretends to connect, apt-get pretends to install, etc.

41. High Interaction Honeypots

42. Client Honeypots Honeymonkey Thug Strider Microsoft Research project.
Network of VMs running IE crawling the web in search of malicious sites that attempt to exploit browsers and distribute malware.
Multiple versions of Windows and IE used.
Thug
Low interaction client honeypot.
Emulates multiple browsers and OSes.

43. Honeytokens
A honeytoken is data that is designed solely for attackers to abuse. Any access to the data is an indication of unauthorized use.
Attempts to download honeytoken files or database records can be identified by NIDS.
Medical record systems will sometimes create fake records for celebrities and politicians.
Mailing lists may contain addresses published nowhere else that point to accounts that accept mail and record sender information.
Maps contain fake streets, towns, or islands to identify when competitors copy the map.

44. Key Points Major models of IDS:
Anomaly detection: unexpected events (statistical IDS.)
Misuse detection: violations of policy (rule-based IDS).
IDS Architecture: sensors, manager, notifier.
Host vs. Network IDS
Host: agent on host checks files, processes to detect attacks.
Network: sniffs and analyzes packets to detect intrusions.
IPS
Stop intrusions, but what about false positives?
Inline vs. non-inline: how do prevention techniques differ?
IDS/IPS Evasion
Honypots

45. References
Richard Bejtlich, The Tao of Network Security Monitoring, Addison-Wesley, 2004.
William Cheswick, Steven Bellovin, and Avriel Rubin, Firewalls and Internet Security, 2nd edition, 2003.
Goodrich and Tammasia, Introduction to Computer Security, Pearson, 2011.
The Honeynet Project, Know Your Enemy, 2nd edition, Addison-Wesley, 2004.
Richard A. Kemmerer and Giovanni Vigna, “Intrusion Detection: A Brief History and Overview,” IEEE Security & Privacy, v1 n1, Apr 2002, pp
Steven Northcutt and Julie Novak, Network Intrusion Detection, 3rd edition, New Riders, 2002.
Michael Rash et. al., Intrusion Prevention and Active Response, Syngress, 2005.
Rafiq Rehman, Intrusion Detection Systems with Snort: Advanced IDS Techniques Using Snort, Apache, MySQL, PHP, and ACID, Prentice Hall, 2003.
Ed Skoudis, Counter Hack Reloaded 2/e, Prentice Hall, 2006.

46. Released under CC BY-SA 3.0
This presentation is released under the Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) license
You are free:
to Share — to copy and redistribute the material in any medium
to Adapt— to remix, build, and transform upon the material
to use part or all of this presentation in your own classes
Under the following conditions:
Attribution — You must attribute the work to James Walden, but cannot do so in a way that suggests that he endorses you or your use of these materials.
Share Alike — If you remix, transform, or build upon this material, you must distribute the resulting work under this or a similar open license.
Details and full text of the license can be found at