1 Issues in Benchmarking Intrusion Detection Systems Marcus J. Ranum.

Slides:



Advertisements
Similar presentations
Presented by Nikita Shah 5th IT ( )
Advertisements

Intrusion Detection Systems (I) CS 6262 Fall 02. Definitions Intrusion Intrusion A set of actions aimed to compromise the security goals, namely A set.
Scalable Parallel Intrusion Detection Fahad Zafar Advising Faculty: Dr. John Dorband and Dr. Yaacov Yeesha 1 University of Maryland Baltimore County.
Insertion, Evasion and Denial of Service: Eluding Network Intrusion Detection Aaron Beach Spring 2004.
1 Chapter 7 Intrusion Detection. 2 Objectives In this chapter, you will: Understand intrusion detection benefits and problems Learn about network intrusion.
Tracking the Role of Adversaries in Measuring Unwanted Traffic Mark Allman(ICSI) Paul Barford(Univ. Wisconsin) Balachander Krishnamurthy(AT&T Labs - Research)
1. AGENDA History. WHAT’S AN IDS? Security and Roles Types of Violations. Types of Detection Types of IDS. IDS issues. Application.
Guide to Network Defense and Countermeasures Second Edition
1.  To analyze and explain the IDS placement in network topology  To explain the relationship between honey pots and IDS  To explain, analyze and evaluate.
Intrusion Detection Systems and Practices
Chapter 10 Firewalls. Introduction seen evolution of information systems now everyone want to be on the Internet and to interconnect networks has persistent.
Snort - an network intrusion prevention and detection system Student: Yue Jiang Professor: Dr. Bojan Cukic CS665 class presentation.
5/1/2006Sireesha/IDS1 Intrusion Detection Systems (A preliminary study) Sireesha Dasaraju CS526 - Advanced Internet Systems UCCS.
IDS Colloquium 2001John Kristoff - DePaul University1 Intrusion Detection Systems (IDS) John Kristoff DePaul University.
Exam ● On May 15, at 10:30am in this room ● Two hour exam ● Open Notes ● Will mostly cover material since Exam 2 ● No, You may not take it early.
Beyond the perimeter: the need for early detection of Denial of Service Attacks John Haggerty,Qi Shi,Madjid Merabti Presented by Abhijit Pandey.
Lesson 13-Intrusion Detection. Overview Define the types of Intrusion Detection Systems (IDS). Set up an IDS. Manage an IDS. Understand intrusion prevention.
Firewalls CS591 Topics in Internet Security November Steve Miskovitz, Steve Peckham, Kan Hayashi.
Intrusion Detection Systems. Definitions Intrusion –A set of actions aimed to compromise the security goals, namely Integrity, confidentiality, or availability,
Host Intrusion Prevention Systems & Beyond
Intrusion Detection - Arun Hodigere. Intrusion and Intrusion Detection Intrusion : Attempting to break into or misuse your system. Intruders may be from.
Intrusion Detection Systems CS391. Overview  Define the types of Intrusion Detection Systems (IDS).  Set up an IDS.  Manage an IDS.  Understand intrusion.
Department Of Computer Engineering
Intrusion Detection System Marmagna Desai [ 520 Presentation]
Building Survivable Systems based on Intrusion Detection and Damage Containment Paper by: T. Bowen Presented by: Tiyseer Al Homaiyd 1.
NET-REPLAY: A NEW NETWORK PRIMITIVE Ashok Anand Aditya Akella University of Wisconsin, Madison.
USENIX LISA ‘99 Conference © Copyright 1999, Martin Roesch Snort - Lightweight Intrusion Detection for Networks Martin Roesch.
1 Intrusion Detection Systems. 2 Intrusion Detection Intrusion is any use or attempted use of a system that exceeds authentication limits Intrusions are.
Computer Security: Principles and Practice First Edition by William Stallings and Lawrie Brown Lecture slides by Lawrie Brown Chapter 8 – Denial of Service.
COEN 252 Computer Forensics
1 Root-Cause Network Troubleshooting Optimizing the Process Tim Titus CTO, PathSolutions.
1 IP: putting it all together Part 2 G53ACC Chris Greenhalgh.
IDS Intrusion Detection Systems CERT definition: A combination of hardware and software that monitors and collects system and network information and analyzes.
Network security Further protocols and issues. Protocols: recap There are a few main protocols that govern the internet: – Internet Protocol: IP – Transmission.
Performance Concepts Mark A. Magumba. Introduction Research done on 1058 correspondents in 2006 found that 75% OF them would not return to a website that.
COEN 252 Computer Forensics Collecting Network-based Evidence.
Intrusion Detection Presentation : 2 OF n by Manish Mehta 02/07/03.
SNORT Feed the Pig Vicki Insixiengmay Jon Krieger.
CSCI 530 Lab Intrusion Detection Systems IDS. A collection of techniques and methodologies used to monitor suspicious activities both at the network and.
Test Loads Andy Wang CIS Computer Systems Performance Analysis.
1 The Internet and Networked Multimedia. 2 Layering  Internet protocols are designed to work in layers, with each layer building on the facilities provided.
High-speed IDS The search for the Holy Grail….. Agenda The Problem Types of IDS’ The Problem Drawbacks Testing Assumptions Conclusions.
1 Impact of IT Monoculture on Behavioral End Host Intrusion Detection Dhiman Barman, UC Riverside/Juniper Jaideep Chandrashekar, Intel Research Nina Taft,
Breno de MedeirosFlorida State University Fall 2005 Network Intrusion Detection Systems Beyond packet filtering.
By Jim White WiredCity, Div. of OSIsoft Copyright c 2004 OSIsoft Inc. All rights reserved. Cyber Security Tools.
Networking Components Daniel Rosser LTEC Network Hub It is very difficult to find Hubs anymore Hubs sends data from one computer to all other computers.
Chapter 5: Implementing Intrusion Prevention
Computer Network Forensics Lecture 6 – Intrusion Detection © Joe Cleetus Concurrent Engineering Research Center, Lane Dept of Computer Science and Engineering,
SNORT Biopsy: A Forensic Analysis on Intrusion Detection System By Asif Syed Chowdhury.
Securing the Network Infrastructure. Firewalls Typically used to filter packets Designed to prevent malicious packets from entering the network or its.
4/19/20021 TCPSplitter: A Reconfigurable Hardware Based TCP Flow Monitor David V. Schuehler.
Denial of Service Attack 발표자 : 전지훈. What is Denial of Service Attack?  Denial of Service Attack = DoS Attack  Service attacks on a Web server floods.
1 Figure 10-4: Intrusion Detection Systems (IDSs) IDSs  Event logging in log files  Analysis of log file data  Alarms Too many false positives (false.
1 Root-Cause VoIP Troubleshooting Optimizing the Process Tim Titus CTO, PathSolutions.
INTRUSION DETECTION SYSYTEM. CONTENT Basically this presentation contains, What is TripWire? How does TripWire work? Where is TripWire used? Tripwire.
Attacking on IPv6 W.lilakiatsakun Ref: ipv6-attack-defense-33904http://
DoS/DDoS attack and defense
Intrusion Detection System
Network Security Terms. Perimeter is the fortified boundary of the network that might include the following aspects: 1.Border routers 2.Firewalls 3.IDSs.
Role Of Network IDS in Network Perimeter Defense.
TCP/IP1 Address Resolution Protocol Internet uses IP address to recognize a computer. But IP address needs to be translated to physical address (NIC).
IDS Intrusion Detection Systems CERT definition: A combination of hardware and software that monitors and collects system and network information and analyzes.
Network Devices and Firewalls Lesson 14. It applies to our class…
Test Loads Andy Wang CIS Computer Systems Performance Analysis.
KAPLAN SCHOOL OF INFORMATION SYSTEMS AND TECHNOLOGY Intrusion Detection and Incidence Response Course Name – IT Intrusion Detection and Incidence.
An Introduction To Gateway Intrusion Detection Systems Hogwash GIDS Jed Haile Nitro Data Systems.
Some Great Open Source Intrusion Detection Systems (IDSs)
IDS Intrusion Detection Systems
Snort – IDS / IPS.
Principles of Computer Security
Presentation transcript:

1 Issues in Benchmarking Intrusion Detection Systems Marcus J. Ranum

2 IDS Benchmarking? How hard can it be to benchmark intrusion detection systems? –Very! –There are lots of ways to get it wrong Accidentally Deliberately –Avoiding doing it wrong does not necessarily mean you’ve done it right

3 What’s an IDS? IDS = Intrusion Detection System –Primary criterion for measurement is the IDS’ ability to detect intrusions –Secondary criteria for measurement are other issues: False positives - false alarms False negatives - real attacks that are missed Performance impact - thruoughput delay or CPU usage on host processor

4 Types of IDS Primary Types: –Network IDS (NIDS) –Host IDS (HIDS) Hybrid Types: –Per-Host Network IDS (PH-NIDS) –Load Balanced Network IDS (LB-NIDS) –Firewall IDS (FW-IDS)

5 Properties of: Network IDS Collect packets in promiscuous mode Issues: –Packet collection rate - what is the maximum throughput? –Reassembly/defragmentation/reordering - what about traffic spoofing? –Selective analysis - is the IDS choosing to ignore some traffic in order to optimize?

6 Properties of: Host IDS Operate on host logs and processes –Sometimes forwards audit records to a central for analysis Issues: –CPU usage on host –What about packet-oriented attacks? –Per-platform (individual) view of attacks - single system is monitored per agent

7 Properties of: Per-Host Network IDS Network IDS “shim” layer inserted into network stack on each host Issues: –Has properties of a network IDS –But: Traffic is processed per-host only Does not have same performance as NIDS “Local” only view of traffic (but no drops)

8 Properties of: Load-Balanced Network IDS Use a load-balancing pre-processor to “spread” load across multiple NIDS Issues: –Can scale to “infinite” bandwidth –Total cost of solution is not single unit pricing (requires switch + multiple NIDS)

9 Properties of: Firewall IDS Place network IDS capability in a firewall or bridge type device Issues: –No packet loss issues (retransmits take care of packets that are lost) –(May) slow down network throughput

10 Other Issues Other things affecting speed and detection ability: –TCP fragment re-assembly –TCP packet re-ordering –TCP state/sequence tracking –Analyzing only selected sessions

11 Fragment Re-assembly Re-assembling fragments takes significant CPU time as well as memory to buffer packets –IDS can be negatively impacted by faked fragments intended to consume extra memory –How does IDS handle fragmented attacks? Simply alert “I see fragmented traffic” or de-fragment then apply IDS logic?

12 Packet Re-ordering Re-ordering packets requires significant CPU as well as memory for packet buffering –IDS can be impacted by unintentional or deliberate packet drops since it tries to buffer out-of-sequence packets –How does IDS handle re-ordering? Does it just flag out-of-sequence packets, or does it re-order then apply IDS logic?

13 TCP State Tracking Tracking TCP states requires maintaining per-session information –IDS is impacted by number of simultaneous streams –IDS is impacted by randomized traffic –IDS is harder to fool with faked out-of- sequence FIN packets

14 Analyzing Selected Sessions IDS can “optimize” performance by only reassembling or tracking TCP related with known signatures –IDS might have extremely good performance against random traffic but poor performance against (e.g.) Web traffic –Tradeoff is coverage versus performance; vendors do not usually document this

15 Naïve Simulation Network Test Network Attack Generator Target Host Attack Stream NIDS

16 What’s Wrong? The Naïve test network permits traffic that is not likely to be seen in a “real world” deployment - e.g.: ARP cache poisoning (you see a lot of this on DEFCON CTF networks) The presence of a router would “smooth” spikes somewhat and actually achieve higher sustained loads

17 Naïve Simulation Network #2 Test Network #2 Target Host Attack Stream NIDS Router w/some screening Test Network #1 Attack Generator Smartbits Load Generator

18 What’s Wrong? SmartBits style traffic generators do not generate “real” TCP traffic –This penalizes IDS that actually look at streams and try to reassemble them (which are desirable properties of a good IDS)

19 Skunking a Benchmark Test Network Attack Generator Target Host w/Host-Net Attack Stream Target Host w/Host-Net Target Host w/Host-Net Smartbits Load Generator

20 What’s Wrong? Packet style counts are not relevant to host-network IDS

21 Skunking a Benchmark: #2 Test Network Attack Generator Target Host Attack Stream Smartbits Load Generator NIDS with selective detection turned on

22 What’s Wrong? IDS with selective detection can be configured to only look at traffic aimed to local subnet –SmartBits style generators’ random traffic largely gets seen and discarded

23 Effective Simulation Network Test Network Replayed packets dumped back onto network NIDSRecorded attack and normal traffic on hard disk

24 What’s Wrong? Nothing: –Predictable baseline –Can verify traffic rate with simple math –Can scale load arbitrarily (use multiple machines each with different capture data) –Traffic is real including “real” data contents –NID cannot be configured to watch a specific machine (there are no targets)

25 Tools to Use Fragrouter - generates fragmented packets Whisker - generates out-of-sequence packets Pcap-pace - replays packets from a hard disk with original inter-packet timing

26 Summary It’s easy to skunk an intrusion detection benchmark It’s hard to design a good intrusion detection benchmark If you want to see if a given system works, the best way to find out is to try it on your actual network