1 Intrusion Detection & Network Forensics Marcus J. Ranum Chief Technology Officer Network Flight Recorder, Inc.

2 An ounce of prevention is worth a pound of detection

3 Why Talk about IDS? Emerging new technology –Very interesting...but... –About to be over-hyped Being informed is the best weapon in the security analyst’s arsenal –It also helps keep vendors honest!

4 What is an Intrusion?! Difficult to define –Not everyone agrees –This is a big problem How about someone telnetting your system? –And trying to log in as “root”? What about a ping sweep? What about them running an ISS scan? What about them trying phf on your webserver? –What about succeeding with phf and logging in?

5 What is IDS? The ideal Intrusion Detection System will notify the system/network manager of a successful attack in progress: –With 100% accuracy –Promptly (in under a minute) –With complete diagnosis of the attack –With recommendations on how to block it …Too bad it doesn’t exist!!

6 Objectives: 100% Accuracy and 0% False Positives A False Positive is when a system raises an incorrect alert –“The boy who cried ‘wolf!’” syndrome 0% false positives is the goal –It’s easy to achieve this: simply detect nothing 0% false negatives is another goal: don’t let an attack pass undetected

7 Objectives: Prompt Notification To be maximally accurate the system may need to “sit on” information for a while until all the details come in –e.g.: Slow-scan attacks may not be detected for hours –This has important implications for how “real-time” IDS can be! –IDS should notify user as to detection lag

8 Objectives: Prompt Notification (cont) Notification channel must be protected –What if attacker is able to sever/block notification mechanism? –An IDS that uses to notify you is going to have problems notifying you that your server is under a denial of service attack!

9 Objectives: Diagnosis Ideally, an IDS will categorize/identify the attack –Few network managers have the time to know intimately how many network attacks are performed This is a difficult thing to do –Especially with things that “look weird” and don’t match well-known attacks

10 Objectives: Recommendation The ultimate IDS would not only identify an attack, it would: –Assess the target’s vulnerability –If the target is vulnerable it would notify the administrator –If the vulnerability has a known “fix” it would include directions for applying the fix This requires huge, detailed knowledge

11 IDS: Pros A reasonably effective IDS can identify –Internal hacking –External hacking attempts Allows the system administrator to quantify the level of attack the site is under May act as a backstop if a firewall or other security measures fail

12 IDS: Cons IDS’ don’t typically act to prevent or block attacks –They don’t replace firewalls, routers, etc. If the IDS detects trouble on your interior network what are you going to do? –By definition it is already too late

13 Paradigms for Deploying IDS Attack Detection Intrusion Detection

14 Internal Network Internet Router w/some screening Firewall DMZ Network WWW Server Desktop Attack Detection IDS detects (and counts) attacks against the Web Server and firewall IDS

15 Attack Detection Placing an IDS outside of the security perimeter records attack level –Presumably if the perimeter is well designed the attacks should not affect it! –Still useful information for management (“we have been attacked 3,201 times this month…) –Prediction: AD Will generate a lot of noise and be ignored quickly

16 Internal Network Internet Router w/some screening Firewall DMZ Network WWW Server Desktop Intrusion Detection IDS detects hacking activity WITHIN the protected network, incoming or outgoing IDS

17 Intrusion Detection Placing an IDS within the perimeter will detect instances of clearly improper behavior –Hacks via backdoors –Hacks from staff against other sites –Hacks that got through the firewall When the IDS alarm goes off, it’s a red alert

18 Attack vs Intrusion Detection Ideally do both Realistically, do ID first then AD –Or, deploy AD to justify security effort to management, then deploy ID (more of a political problem than a technical one) The real question here is one of staffing costs to deal with alerts generated by AD systems

19 IDS Data Source Paradigms Host Based Network Based

20 Host Based IDS Collect data usually from within the operating system –C2 audit logs –System logs –Application logs Data collected in very compact form –But application / system specific

21 Host Based: Pro Quality of information is very high –Software can “tune” what information it needs (e.g.: C2 logs are configurable) –Kernel logs “know” who user is Density of information is very high –Often logs contain pre-processed information (e.g.: “badsu” in syslog)

22 Host Based: Con Capture is often highly system specific –Usually only 1, 2 or 3 platforms are supported (“you can detect intrusions on any platform you like as long as it’s Solaris or NT!”) Performance is a wild-card –To unload computation from host logs are usually sent to an external processor system

23 Host Based: Con (cont) Hosts are often the target of attack –If they are compromised their logs may be subverted –Data sent to the IDS may be corrupted –If the IDS runs on the host itself it may be subverted

24 Network Based IDS Collect data from the network or a hub / switch –Reassemble packets –Look at headers Try to determine what is happening from the contents of the network traffic –User identities, etc inferred from actions

25 Network Based: Pro No performance impact More tamper resistant No management impact on platforms Works across O/S’ Can derive information that host based logs might not provide (packet fragmenting, port scanning, etc.)

26 Network Based: Con May lose packets on flooded networks May mis-reassemble packets May not understand O/S specific application protocols (e.g.: SMB) May not understand obsolete network protocols (e.g.: anything non-IP) Does not handle encrypted data

27 IDS Paradigms Anomaly Detection - the AI approach Misuse Detection - simple and easy Burglar Alarms - policy based detection Honey Pots - lure the hackers in Hybrids - a bit of this and that

28 Anomaly Detection Goals: –Analyse the network or system and infer what is normal –Apply statistical or heuristic measures to subsequent events and determine if they match the model/statistic of “normal” –If events are outside of a probability window of “normal” generate an alert (tuneable control of false positives)

29 Anomaly Detection (cont) Typical anomaly detection approaches: –Neural networks - probability-based pattern recognition –Statistical analysis - modelling behavior of users and looking for deviations from the norm –State change analysis - modelling system’s state and looking for deviations from the norm

30 Anomaly Detection: Pro If it works it could conceivably catch any possible attack If it works it could conceivably catch attacks that we haven’t seen before –Or close variants to previously-known attacks Best of all it won’t require constantly keeping up on hacking technique

31 Anomaly Detection: Con Current implementations don’t work very well –Too many false positives/negatives Cannot categorize attacks very well –“Something looks abnormal” –Requires expertise to figure out what triggered the alert –Ex: Neural nets can’t say why they trigger

32 Anomaly Detection: Examples Most of the research is in anomaly detection –Because it’s a harder problem –Because it’s a more interesting problem There are many examples, these are just a few –Most are at the proof of concept stage

33 Misuse Detection Goals: –Know what constitutes an attack –Detect it

34 Misuse Detection (cont) Typical misuse detection approaches: –“Network grep” - look for strings in network connections which might indicate an attack in progress –Pattern matching - encode series of states that are passed through during the course of an attack e.g.: “change ownership of /etc/passwd ” -> “open /etc/passwd for write” -> alert

35 Misuse Detection: Pro Easy to implement Easy to deploy Easy to update Easy to understand Low false positives Fast

36 Misuse Detection: Con Cannot detect something previously unknown Constantly needs to be updated with new rules Easier to fool

37 Burglar Alarms A burglar alarm is a misuse detection system that is carefully targeted –You may not care about people port- scanning your firewall from the outside –You may care profoundly about people port-scanning your mainframe from the inside –Set up a misuse detector to watch for misuses violating site policy

38 Burglar Alarms (cont) Goals: –Based on site policy alert administrator to policy violations –Detect events that may not be “security” events which may indicate a policy violation New routers New subnets New web servers

39 Burglar Alarms (cont) Trivial burglar alarms can be built with tcpdump and perl Netlog and NFR are useful event recorders which may be used to trigger alarms ftp://coast.cs.purdue.edu/pub/tools/unix/netlog/

40 Burglar Alarms (cont) The ideal burglar alarm will be situated so that it fires when an attacker performs an action that they normally would try once they have successfully broken in –Adding a userid –Zapping a log file –Making a program setuid root

41 Burglar Alarms (cont) Burglar alarms are a big win for the network manager: –Leverage local knowledge of the local network layout –Leverage knowledge of commonly used hacker tricks

42 Burglar Alarms: Pro Reliable Predictable Easy to implement Easy to understand Generate next to no false positives Can (sometimes) detect previously unknown attacks

43 Burglar Alarms: Con Policy-directed –Requires knowledge about your network –Requires a certain amount of stability within your network Requires care not to trigger them yourself

44 Honey Pots A honey pot is a system that is deliberately named and configured so as to invite attack –swift-terminal.bigbank.com –www-transact.site.com –source-r-us.company.com –admincenter.noc.company.net

45 Honey Pots (cont) Goals: –Make it look inviting –Make it look weak and easy to crack –Instrument every piece of the system –Monitor all traffic going in or out –Alert administrator whenever someone accesses the system

46 Honey Pots (cont) Trivial honey pots can be built using tools like: –tcpwrapper –Burglar alarm tools (see “burglar alarms”) –restricted/logging shells (sudo, adminshell) –C2 security features (ugh!) See Cheswick’s paper “An evening with Berferd” for examples

47 Honey Pots: Pro Easy to implement Easy to understand Reliable No performance cost

48 Honey Pots: Con Assumes hackers are really stupid –They aren’t

49 Hybrid IDS The current crop of commercial IDS are mostly hybrids –Misuse detection (signatures or simple patterns) –Expert logic (network-based inference of common attacks) –Statistical anomaly detection (values that are out of bounds)

50 Hybrid IDS (cont) At present, the hybrids’ main strength appears to be the misuse detection capability –Statistical anomaly detection is useful more as backfill information in the case of something going wrong –Too many false positives - many sites turn anomaly detection off

51 Hybrid IDS (cont) The ultimate hybrid IDS would incorporate logic from vulnerability scanners* –Build maps of existing vulnerabilities into its logic of where to watch for attacks Backfeed statistical information into misuse detection via a user interface * Presumably, a clueful network admin would just fix the vulnerabilty

52 Books Internet Security and Firewalls: Repelling the Wily Hacker, by Bill Cheswick and Steve Bellovin, from Addison Wesley Internet Firewalls, by Brent Chapman and Elizabeth Zwicky

53 URLs Spaf’s Security Page – Mjr’s home page – Hacker sites: the fringe – –

54 Addresses CERT Firewalls mailing list subscribe firewalls Web security mailing list subscribe www-security

55 Addresses Firewalls Wizards mailing list subscribe firewall- wizards –Searchable online archive on