Towards Vulnerability-Based Intrusion Detection with Event Processing Amer Farroukh, Mohammad Sadoghi, Hans-Arno Jacobsen University of Toronto July 13, 2011 DEBS'11

Limitation of Regular Expressions Conficker worm infected more than 10 million hosts in Economic loss tallied up to $9.1 billion. July 13, 2011DEBS'11 IDS Attacker RE-Signature bin/*sh bin/sh binbin/sh bin//sh UofT Network bin/delete

Signature-based IDSes Exploit-based (Snort/ Cisco /Proventia): Regular Ex Vulnerability-based: Leverage protocol semantics Complex signatures: Multiple PDUs (ex. Conficker) July 13, 2011DEBS'11 Buffer overflow (all exploits) Filename ="login.htm" && len(uri.assignment_sequence.variable["password"])>20 Buffer overflow (shellcode) content: "|74 07 eb|“ && distance: 1 && within: 1 && pcre: "/\xeb.[\x58-\x5b]\x31[\xc9\xd2 \xdb]/bin/sh” Buffer overflow after binding to server BIND PDU: ver=3.0 && UUID=“4b324fc d a47bf6ee188” ACK PDU: ver=3.0 && result[UUID] = Accept REQ PDU: ver=3.0 && opnum=0x1f && strlen(stub.PathName)> 256 && matchRE(stub.PathName, “/^\x05\x00\x00”)

Outline Related Work & System Architecture Matching Algorithms – Access Predicate Pruning (APP) – Early Elimination (EE) Multiple Protocol Data Units (MPDU) Support – Memory Conscious Network (MCN) Experimental Evaluations Conclusions July 13, 2011DEBS'11

Related Work Vulnerability-based signature matching Evaluate signatures over a stream data packets – High-speed matching [RAID’08] Programmer has to hard code signatures into the parser. – Candidate Selection (CS) [SIGCOMM‘10] Only algorithm proposed in IDS to match many signatures Re-compute candidate list for every field parsed Event processing (Publish/Subscribe Matching) Evaluate subscriptions (signatures) over a stream of events (packets) – Propagation [SIGMOD’01] Targets specific type of predicates – Counting [ACM TODS‘94] Predicate matching and signature matching are distinct. Can support arbitrary matchers – BE-Tree [SIGMOD’11] (EPTS Principle Award) Two-phase space-cutting to iteratively refine and prune the search space July 13, 2011DEBS'11

Event Processing vs. IDS July 13, 2011DEBS'11 MetricEvent ProcessingIDS WorkloadDynamic (subs constantly enter and leave the system) Static (DS torn down and rebuilt when a new signature is added) ParsingMessages are parsed before they are passed to broker Parsing is crucial to enhancing performance Matching ProbabilityLarge number of subs are matched Signatures are rarely matched Memory Clean-upPartial matches may reside in the system for an extended time Memory per connection must be minimal

Our Contribution Multiple PDU Component (MCN) Parser Generator Signature Compiler Protocol Specs Vulnerability Signature set Individual Matchers (e.g., String, RE Matchers) Matching Algorithm M1M2M3M4M5 APPEE Traffic Capture (Libpcap) Traffic Capture (Libpcap) TCP Reassembly (Libnids) TCP Reassembly (Libnids) Protocol Identification (Port or PIA_Bro) Protocol Identification (Port or PIA_Bro) Leverage Existing Systems Packets StubPAC IDL File & Signatures Netshield Core Engine Protocol Parser (Minimal) Protocol Parser (Minimal) System Architecture July 13, 2011 DEBS'11

Outline Related Work & System Architecture Matching Algorithms – Access Predicate Pruning (APP) – Early Elimination (EE) Multiple Protocol Data Units (MPDU) Support – Memory Conscious Network (MCN) Experimental Evaluations Conclusions July 13, 2011DEBS'11

Predicate List Access Predicate List Ap1P1P2Ap2P3P4P5Ap3P6P7ApNPiPjPk SNSN 4 S3 3 S2 4 S1 3 S2S3SN Pre-computation Phase Partial Matches Add to List SiSi Index Counter CN 4 C3 1 C2 3 C1 1 SjSj Create Counter Cj 1 Runtime Signature Matching Check Counters String Matcher Access Predicate List Predicate List Number Matcher Access Predicate List Predicate List Length Matcher Access Predicate List Predicate List Range Matcher Access Predicate List Predicate List RE Matcher Access Predicate List Predicate List Predicate Type PiPi Runtime Predicate Matching Access Predicate Pruning (APP) Access Predicate Signature Predicate S N is matched

Access Predicate List Partial Matches Add to List SjSj Create Counter Cj 1 Runtime Signature Matching CN 4 C3 1 C2 3 C1 1 Check Counters Signature Compilation S id Increases S1 S2 S4 S5 S9 Predicate List Dual Scan Increment Counter (If Matched) Early Elimination (EE) Ap1P1P2Ap2P3P4P5Ap3P6P7ApNPiPjPk SNSN 4 S3 3 S2 4 S1 3 S2S3SN Pre-computation Phase S N is matched

APP and EE Evaluation July 13, 2011DEBS'11

AP Selectivity 12

Outline Related Work & System Architecture Matching Algorithms – Access Predicate Pruning (APP) – Early Elimination (EE) Multiple Protocol Data Units (MPDU) Support – Memory Conscious Network (MCN) Experimental Evaluations Conclusions July 13, 2011DEBS'11

MPDU Signatures: S4=S2&S3 S5=S1->S2 S6=S1&(S2&S3) S7=(S1||S2)&S3 S1S2S3 00 & 00 & Sample run: S1S3S2 10 & 01&01& 11 & 11 & S4 S7 S5S6 Output: S7 || S5 S4 0 -> 00 & S7 S6 00 & HASH Si 00 & JN1JN2 JN3 JN4JN5 Memory Conscious Network (MCN) Signature Nodes Join Nodes

MCN Evaluation AlgorithmSequentialMCN Signature Nodes AND Nodes8058 NEXT Nodes8568 OR Nodes3020 Memory per connection (bytes) 3124 July 13, 2011DEBS'11

Conclusions and Future Work Vulnerability-based signature matching – Proposed two novel solutions APP and EE – Attack resilient and faster than CS – Access predicate selectivity (future work) MPDU support – One of the first efforts to match MPDU signatures – MCN is memory efficient and 29 times faster than sequential scan – Balancing network depth and node sharing (future work) July 13, 2011DEBS'11

July 13, 2011DEBS'11

Challenges of Vulnerability Signatures Enable high speed parsing – Parse only relevant fields Support arbitrary matchers – RE, strings, length-checking, numbers, and ranges Reduce state maintenance – Avoid state explosion for MPDU matching July 13, 2011DEBS'11

Time Complexity (Worst Case) – APP For every predicate: O(Predicate List + AP List) Final Scan: O(Partial Matches List) – EE For every predicate: O(Predicate List + Partial Matches AP List) Final Scan: O(Partial Matches List) Memory Footprint (APP & EE) – Determined by size of Partial Matches List July 13, 2011DEBS'11 APP and EE Complexities