Language-Based Generation and Evaluation of NIDS Signatures Shai Rubin Somesh Jha Barton P. Miller University of Wisconsin, Madison

Rubin, Jha, Miller2 Attacker “TYPE A \n CWD \n” Network NIDS Signature database Misuse Network Intrusion Detection System (NIDS) Problem: A single attack might have many forms: –Ptacek and Newsham, 1988 –Handley and Paxson, 2001 –Marty, 2002 –Mutz, Vigna, and Kemmerer, 2003 –Vigna, Robertson, and Balzarotti, 2004 –Rubin, Jha, Miller, 2004 –And others... “TYPE A \n (.)* CWD ” TYPE A \n LIST \n CWD...

Rubin, Jha, Miller3 Attacker Network NIDS Signature database Problem: Accurate Signatures Today, we construct signatures in an ad-hoc manner Challenges: complex protocols, redundancy Questions: –Can we systematically construct an accurate signature? –Can we systematically evaluate a signature? –Can we systematically compare signatures? “TYPE A \n (.)* CWD ” TYPE A \n LIST \n CWD...

Rubin, Jha, Miller4 Contributions Practical: provide signature writers a methodology and a tool that enables them to systematically construct a signature, evaluate its accuracy, and compare it to other signatures Conceptual: –a session signature, –a semantic model for an attack protocol, –a language-base approach for signature construction

Rubin, Jha, Miller5 A NIDS Signature Attack: a set of TCP streams Signature: a set of TCP streams TCP Streams ASig

Rubin, Jha, Miller6 A NIDS Signature Attack: a set of TCP streams Signature: a set of TCP streams A prefect signature: Sig=A TCP Streams ASig Sig=A

Rubin, Jha, Miller7 A NIDS Signature Attack: a set of TCP streams Signature: a set of TCP streams A prefect signature: Sig=A Problem: most of the time A is unknown. Difficult to: –construct accurate a signature –evaluate changes to the signature –compare signatures TCP Streams A Sig

Rubin, Jha, Miller8 A NIDS Signature TCP Streams A Sig Attack: a set of TCP streams Signature: a set of TCP streams A prefect signature: Sig=A Problem: most of the time A is unknown. Difficult to: –construct accurate a signature –evaluate changes to the signature –compare signatures

Rubin, Jha, Miller9 Language-Based Approach TCP Streams Attack: the language A ghost Signature: the language L sig Goal: compare the language Problem: difficult to determine containment  A ghost. Ideas: 1.Abstraction: over-approximate A ghost, such that it is easy to determine containment 2.Automation: Use an automatic tool to compare L sig and A inv L sig A ghost A inv

Rubin, Jha, Miller10 Language-Based Signature Construction TCP Streams L sig A ghost A inv  ConclusionAction  fp  fn

Rubin, Jha, Miller11 Language-Based Signature Construction TCP Streams L sig A ghost A inv  ConclusionAction L sig  A inv A false positive Shrink signature  fp  fn

Rubin, Jha, Miller12 Language-Based Signature Construction TCP Streams L sig A ghost A inv  fp  ConclusionAction L sig  A inv A false positive Shrink signature  L sig  A inv A inv  fn

Rubin, Jha, Miller13 Language-Based Signature Construction TCP Streams L sig A ghost A inv  fp  ConclusionAction L sig  A inv A false positive Shrink signature  L sig  A inv A false negative Expand signature A inv  fn

Rubin, Jha, Miller14 Language-Based Signature Construction TCP Streams L sig A ghost A inv  fp  ConclusionAction L sig  A inv A false positive Shrink signature  L sig  A inv A false negative Expand signature A spurious sequence Refine A inv A inv  fn  sp

Rubin, Jha, Miller15 Language-Based Signature Construction TCP Streams L sig A ghost A inv  fp  ConclusionAction L sig  A inv A false positive Shrink signature  L sig  A inv A false negative Expand signature A spurious sequence Refine A inv L sig  A inv Discussion in the paper  L sig  A inv A inv  fn  sp

Rubin, Jha, Miller16 Outline Goal: develop methodology to construct and evaluate signatures Main idea: use a formal language to approximate A ghost and automatically compare this language to L sig The languages The signature construction process

Rubin, Jha, Miller17 L sig : A Syntactic Representation of the Attack Our signature is a regular language Alphabet: application-level events. For example, FTP commands A session signature: a string in the language represents the entire attack. Each signature is a concatenation of three languages: preparation (L pre ), exploitation (L exp ), and confirmation (L conf )

Rubin, Jha, Miller18 ftp-cwd [CAN ] Preparation: FTP login login L logout Q QQ LL TokenDescription L Login confirmation Q Connection termination

Rubin, Jha, Miller19 ftp-cwd [CAN ] Preparation: FTP login Exploitation: A CWD command with a long argument login L logout Q QQ LL attack A such that (length>100 && data  (.) * /bin/sh(.) * C login TokenDescription L Login confirmation Q Connection termination C CWD command A CWD argument

Rubin, Jha, Miller20 L ftp-cwd : ftp-cwd Session Signature Non-recursive hierarchical state machine Constructed automatically Can be analyzed intrusion logout2 1attack A,I R,L IRIR A,L C I R,L C A,C,I R,Q Q Q CQL A accept start reject  

Rubin, Jha, Miller21 L ftp-cwd : Vs. Snort Non-recursive hierarchical state machine Constructed automatically Can be analyzed intrusion logout2 1attack A,I R,L IRIR A,L C I R,L C A,C,I R,Q Q Q CQL A accept start reject  

Rubin, Jha, Miller22 Language-Based Signature Construction TCP Streams Session Signature A ghost A inv  fp  ConclusionAction L sig  A inv A false positive Shrink signature  L sig  A inv A false negative Expand signature A spurious sequence Refine A inv L sig  A inv Discussion in the paper  L sig  A inv A inv  fn  sp

Rubin, Jha, Miller23 A inv : Semantic Representation of the Attack Another regular language Models semantics properties: –“Requires FTP login” –“Requires ASCII FTP mode” –“Requires HTTP 1.1” Using an FSM we model the semantics of the application-level protocol that the attack uses

Rubin, Jha, Miller24 FTP Semantic Model VariableDescriptionValues X1X1 User logged in{0,1} X2X2 FTP transfer mode{‘A’,’B’,0} NameTokenDescriptionPrecond.Postcond. SLOGINLVictim indicates successful login-X 1 =1,X 2 =‘A’ BINARYB Attacker issues TYPE B command X 1 =1X 2 =‘B’ ASCIIA Attacker issues TYPE A command X 1 =1X 2 =‘A’ VQUITQ1Q1 Victim terminates connection-  X i =0 UQUITQ2Q2 Attacker terminates connection-  X i =0 FTP State variables FTP Transitions

Rubin, Jha, Miller25 Language-Based Signature Construction TCP Streams Session Signature A ghost Semantic model  fp  fn Semantic Model Signature Spin String/ NULL SP FN or FP Manual refinement (currently) Automatic comparison

Rubin, Jha, Miller26 TCP Streams Constructing a Signature for ftp-cwd login=1 L pre L exp False Positive L1L1 (.) * CWD Semantic Model Signature Spin String/ NULL

Rubin, Jha, Miller27 TCP Streams Constructing a Signature for ftp-cwd login=1 FP 1 L1L1 L pre L exp False Positive L1L1 (.) * CWD FP 1 =“CWD ” Semantic Model Signature Spin String FP 1

Rubin, Jha, Miller28 TCP Streams Constructing a Signature for ftp-cwd login=1 FP 1 L1L1 L pre L exp False Positive L1L1 (.) * CWD FP 1 =“CWD ” L2L2 L(.) * CWD Semantic Model Signature Spin String/ NULL

Rubin, Jha, Miller29 TCP Streams Constructing a Signature for ftp-cwd L1L1 login=1 FP 1 FP 2 L2L2 L pre L exp False Positive L1L1 (.) * CWD FP 1 =“CWD ” L2L2 L(.) * CWD FP 2 =“L  UQUIT  CWD ” Semantic Model Signature Spin String FP 2

Rubin, Jha, Miller30 TCP Streams Constructing a Signature for ftp-cwd login=1 FP 1 L1L1 FP 2 L2L2 L pre L exp False Positive L1L1 (.) * CWD FP 1 =“CWD ” L2L2 L(.) * CWD FP 2 =“L  UQUIT  CWD ” L3L3 L(  UQ) * CWD Semantic Model Signature Spin String/ NULL

Rubin, Jha, Miller31 TCP Streams Constructing a Signature for ftp-cwd login=1 L pre L exp False Positive L1L1 (.) * CWD FP 1 =“CWD ” L2L2 L(.) * CWD FP 2 =“L  UQUIT  CWD ” L3L3 L(  UQ) * CWD FP 3 =“L  VQUIT  CWD ” FP 1 L1L1 FP 2 L2L2 FP 3 L3L3 Semantic Model Signature Spin String FP 3

Rubin, Jha, Miller32 TCP Streams Constructing a Signature for ftp-cwd login=1 L pre L exp False Positive L1L1 (.) * CWD FP 1 =“CWD ” L2L2 L(.) * CWD FP 2 =“L  UQUIT  CWD ” L3L3 L(  UQ) * CWD FP 3 =“L  VQUIT  CWD ” FP 1 L1L1 FP 2 L2L2 FP 3 L3L3 Semantic Model Signature Spin NULL

Rubin, Jha, Miller33 Constructing a Signature for ftp-cwd TCP Streams login=1 FP 1 L1L1 FP 2 L2L2 FP 3 L3L3 L 1  L 2  L 3  L 4 L4L4 More false positivesLess false positives Comparing signature: It is possible to show that L 4 does not miss more attacks than L 1 (under certain assumptions)

Rubin, Jha, Miller34 Constructing a Signature for pro-ftpd Session Signature (simplified)False Negative/Spurious L  TYPEA  ST  RET  RET TCP Streams login=1 TYPE=‘A’

Rubin, Jha, Miller35 Constructing a Signature for pro-ftpd Session Signature (simplified)False Negative L  TYPEA  ST  RET  RETFN 1 =L  ST  RET  RET TCP Streams login=1 TYPE=‘A’ FN 1 Two signatures based on the configuration of the FTP server

Rubin, Jha, Miller36 Lessons to Take Home A methodology to construct and evaluate signatures Able to detect loopholes in signatures, loopholes that we did not anticipate The accuracy of the signature depends of the accuracy of the semantic model TCP Streams Session Signature A ghost A inv  fp A inv  fn  sp