Intrusion Detection Systems Taxonomy and Survey

Papers Intrusion Detection Systems: A survey and Taxonomy. Research in Intrusion Detection Systems: A Survey – Stefan Axelsson Papers have done a survey of various IDS available. Classified those systems in to various categories depending upon the way they work. What are all the works that are pending and need to be addressed in the area of IDS.

IDS - Introduction Detecting inappropriate, incorrect or anomalous activities. Host-based, Network based. Host-Based: Monitoring activities in the host. Network –Based: Monitor network activities. “Burglar alarms”: Site Security Officer (SSO) responds to the alarm. IDS need to detect suspicious activities. Intrusion and Misuse. Intrusion usually are attacks from outside and Misuse happens from inside. Host- based: checking integrity of the system and track down malicious processes. Network-based : Monitoring Traffic on a LAN. A Promiscuous interface sniffing packets passively. Burglar alarms that sound when site’s security is compromised and so that Site Security officer (SSO) can respond

IDS - Components Audit agent, detector proper, SSO Audit Agent: Collects data from the observed entity. Detector proper: Processes the collected data. SSO : Gets input (alarm) from Detector Proper and decides on further investigation/Action. Audit Agent: Collection agent that collects the system under observation. Detector proper either stores or processed by the DP. SSO decides on further actions depending upon the policies.

IDS - Taxonomy Detector Proper : Central to IDS. Intrusion Detection Principles: Intrusion detector, Intrusion that needs to be detected and the environment in which the intrusion is detected. How to formalize intrusion detection decision. Current Research systems are classified based on their implementation mechanisms but not on the detection principles. Detector Proper : focus of research What to detect? : what is the source that needs to be observed.. How: How the decided source needs to be observed? By Security logging. Observing a source has many challenges like background traffic etc.. How to Isolate “bad” stuffs? When: when to raise the alarm? Current research systems are not properly classified based on principles. (loosely classified as Expert system based IDS, ANN based, data mining based)

IDS - Anomaly/Signature/Compound Intrusion Detection decision is based on: Anomaly detection: Abnormalities of traffic in question. “abnormal” is probably suspicious. Signature detection: - Knowledge of intrusions and its traces. Compound detection: - Hybrid of anomaly and signature. Anomaly: Doesn’t watch for known intrusions. Signature: Watches based on known intrusions and their traces Compound: Intrusion decision based on both anomalous behavior and signature of known intrusions. More false alarms!

IDS - Anomaly Detection Detection principle is based on: - What is normal for the subject under observation? - On what percentage of activity to flag abnormal? How to make this particular decision? Anomaly Detection is further classified: - Self-Learning - Programmed. Detection principle flags behavior that is unlikely to originate from the normal process. Anomaly has no regard for the actual intrusion scenario. Example: Research paper that discussed in “Distributed Computing”: Observe the sequence of system calls made by the processes and flag when an abnormal sequence is encountered (probably from a malicious process) Anomaly is further classified depending upon the “normal” behavior learning.

IDS - Anomaly Detection Self Learning systems Learns from Examples of “normal” behavior Rule Modelling: Studies normal traffic and formulates “rules”. Applies the rules and raises alarm if there is deviation. Descriptive Statistics: Creates a profile of statistics of different system parameters. Constructs a distance vector of the observed traffic to the profile and alarm is raised if the deviation is more. Self Learning: Observing the traffic for an extended period of time and building some model of the underlying process. Two stages: Learning phase and detection phase Normal behavior is described by a set of rules. Distance vector is created between the observed traffic and the created profile.

IDS - Anomaly Detection Self Learning systems ANN (Artificial Neural Network) based: The system’s normal traffic is fed in to an ANN and it subsequently learns the pattern. After this “training” phase, new traffic is fed and intrusion detection decisions are taken. Training phase and the production phase. Separate topics on machine learning techniques like ANN, genetic algo in the future talks.

IDS - Anomaly Detection Programmed systems Some one has to teach the system to detect anomalies. User has to determine what behavior is abnormal that should result in signaling security violation. Further divided in to: Descriptive Statistics. Default Deny. Doesn’t self learn. User has to input all the errant , abnormal behavior for which the IDS should raise the flag.

IDS - Programmed systems Descriptive statistics Profile of normal behavior built from descriptive statistics on number of system parameters. - Simple statistics: higher level components use simple stats to arrive at more abstract intrusion detection decisions. - Simple Rule Based: User provides rules to apply on the collected statistics. - Threshold: When the system has collected necessary statistics, the user can program predefined thresholds to determine whether to raise alarm or not. Parameters: number of unsuccessful logins, number of network connections open, number of commands with error return etc. Threshold: Examples include like 3 times try for logins. N number of network connections from a particular host. Rules dependent on security policies. When the behavior of the system crosses the threshold of any measured statistics, signal alarm.

IDS - Programmed systems Default Deny Default Deny: Explicitly state the circumstances under which the system operates in a secure-benign manner, and to flag all deviations from this as intrusive. Based on security policies. State Series Modelling: Policy is encoded as a set of states. State series Modelling: Policy encoded as set of states. A state machine is run and the current state is matched to one of the states and the machine waits for another transition to occur. If a transition occurs to an unspecified state, then IDS sounds the alarm. (file accesses, opening of a secure communication port etc)

IDS - Signature Detection Decision based on the model of intrusion and its traces. Detecting intrusive behavior without idea of the normal behavior or background traffic of the system. Looks for patterns that are suspicious. Intrusion detection decision is formed on the basis of knowledge of a model of the intrusive process and what traces it might leave on the observed system. Doesn’t care about the current state , normal behavior of the system etc.

IDS - Signature Detection Programmed systems Programmed systems: Idea is to determine explicitly the traces of intrusion. Types: 1. State-Modelling: - The intrusion is encoded as states. - The states form a simple chain and all that states must be traversed for the intrusion to be considered as taken place. - The states can form a petrinet with a general tree structure. The system is fed with explicit traces of intrusion that can be thought to occur uniquely in the observation space. State – Modelling: All the states must be present in the observed system for a particular intrusion to take place. Petrinet: Several preparatory states can be fulfilled in any order, irrespective of where in the model they occur.

IDS - Signature Detection Programmed systems 2. Expert-System: Reason about the security state of the system based on rules that describe an intrusive behavior. 3. String-matching: Searching malicious substrings within a long string (audit data). 4. Simple-rule based: Less complex expert systems with faster execution. New decisions are taken or new rules are formed when the given set of rules are true.

IDS – Compound Detectors. Hybrid of Anomaly and signature based detection techniques. Decision is made based on both the normal behavior of the system and the intrusive behavior of the intruder. Signature Inspired. Most advanced! Mostly self learning: The system learns the normal and intrusive behaviors during the training phase. Signature Inspired: Intrusive model is much stronger than the normal model. Most advanced!

IDS - Categories: Type of Intrusions Well Known: Intrusions that are well known and a ‘static’, well defined pattern can be found. Detected by Signature based IDS systems. Generalisable: similar to well-Known with slight variations. Detected by compound systems based on self learning. (RIPPER) Unknown: IDS does not know what to expect. Anomaly detection based IDS may detect. Generalisable: code red1 and code red2. RIPPER can detect varying intrusion signatures.

IDS - Taxonomy: system characteristics Taxonomy based on the approach employed by Intrusion detection systems on the audit data. 1. Time of Detection. (real time, non real-time) 2. Granularity of audit data processing. 3. Source of audit data (network or host). 4. Response to detected intrusions. (Passive vs. Active) (Active Systems : may exercise control on the attacked or attacking system or both ) Time of Detection: Processing the audit data in real time or non-real time. Granularity: continuous data processing or processing in batch. Response to detected intrusions: Passive : responds by notifying to the authority (they don’t take any measures to fix the problem). Active : Responds by exercising control on the attacked system. Try to change the state of the system to thwart Or mitigate the effects of attack. Exercise control over attacking system. Try to remove attacker’s platform. Active systems may block suspicious system calls. Is Prone to DoS attacks though.

IDS - Taxonomy: system characteristics 5. Locus of data processing. (central or distributed) 6. Locus of data-collection. (Central or distributed) 7. Security: IDS security. 8. Interoperability. Paper categorizes the surveyed ID systems in the above categories also.

IDS - Trends Present research on IDS tend more towards: IDS with “Active” type of response. From centralized to Distributed IDS. Security of IDS. (Resistant to attack on IDS itself) From host to network. (Problem with encrypted data)

IDS: Open Research Questions Research done till date fails to answer: Nature of intrusions that the system is trusted to detect. To what degree the IDS classify intrusions? Can it be trusted to respond “actively” to them? What audit data to collect and how to collect, store, prune and transmit effectively? Have we found all possible types of intrusions? Run-time efficiency. Active response is prone to DoS. How to test IDS?

IDS - MIDAS First Signature based detection IDS. Has a “rules” database. Introduction of a new rule triggers reevaluation of existing rules and this in turn will introduce additional new rules. Rule database is populated with rules from different categories (User anomaly, immediate attack, system state). User anomaly: statistical profiles of the previous user behavior to detect deviations. Immediate attacks: Obvious attacks. Need not worry about the system state and configuration. System state: stats of system without concern for users.

IDS - IDES Anomaly based intrusion detection. Motivation is “users behave in a consistent manner” IDES monitors various parameters for the user behavior (CPU usage, Command usage, Network activity etc) and builds detection rules.