1. High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department of Computer Science † Department of Electrical and Computer Engineering {nschear2, dalbrech, nikita}@illinois.edu 16 September 2008

2. 2 Exploit vs. Vulnerability Signatures Exploit Signatures – Match a specific example of an exploit Vulnerability Signatures – Match the condition at which the program is vulnerable + fast to match - imprecise, false positives + exploit generic, very precise - expensive

3. 3 Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked A05 header attribute HTTP IPP extra data attribute

4. 4 Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked A05 header attribute HTTP IPP extra data attribute tagname_lenname value_lenvalue

5. 5 Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked A05 header attribute HTTP IPP extra data attribute tag name_lenname value_lenvalue Buffer overflow: uint16 name_len used to copy name into 8KB buffer without checks

6. 6 Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked A05 header attribute HTTP IPP extra data attribute tag name_len value_lenvalue 0xA190909090EB105B4B33C966B996 0380340BFDE2FAEB05E8EBFFFFFF Exploit Signature alert tcp any any -> any 631 (content: “|EB 10 5B 4B 33 C9 66 B9 96 03…|”) Shell code stored in name field

7. 7 Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked E5 header attribute tag nam e_ len 0xA190909090EB10 5B4B33C966B99 920 extra data attribute value_lenvalue 60380340BFDE2FA EB05E8EBFFFFFF HTTP Chunk 1 Chunk 2 attribute Now split shell code across two HTTP chunks

8. 8 Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked A05 header attribute HTTP IPP extra data attribute tagname_lenname value_lenvalue Vulnerability Signature if(name_len > 8192) Exception!

9. 9 Motivation: Matching Performance Protocolbinpachand-coded CUPS/HTTP5,41420,340 DNS712,647 IPP8097,601 WMF61014,013 Throughput (Mbits/s) of vulnerability matchers Hand-coded 3x to 37x faster! Many vulnerabilities do not require full protocol parsing

10. 10 Introducing VESPA A vulnerability signature and protocol parsing architecture Focus on performance –Hardware acceleration friendly design Future work: Offload to FPGA, network processor –Target use in NIC or switch 1 Gbps+ Low latency

11. 11 Outline Parsing Architecture Design –Text Protocols –Binary Protocols Vulnerability Specification Language Performance Evaluation Related Work Conclusions

12. 12 VESPA Design Couple protocol and vulnerability specifications –maximum parser optimization Design Principles –Fast matching primitives –Explicit State Management –Avoid parsing irrelevant message parts Basic Idea: Construct matching specs based on primitives and marry to state control functions

13. 13 Protocol State Core State –Example: HTTP Content-Length header –Define structure and semantics of the message Always parse

14. 14 Protocol State Core State –Example: HTTP Content-Length header –Define structure and semantics of the message Always parse Application State –Example: HTTP Accept-Charset header –Only relevant to the application Skip by default

15. 15 Text Protocols Often use explicit field labeling –e.g., RCPT TO: multi-string matching primitive to flatten irrelevant protocol structure –e.g., search for “HTTP/1.”, “Content-Length:”, “Transfer-Encoding:”, “POST”, and “\r\n\r\n” simultaneously Use control logic to drive matching primitive

16. 16 Binary Protocols Field meaning based on position in message Binary traversal primitive –Parses only core fields –No full in-memory representation –Parses vulnerability relevant fields when desired –Implemented with binpac language

17. 17 VESPA Language Stores each var as a member of generated C++ class Extraction function within %{…}% bool is_post = str_matcher “POST” handler handle_post() %{ is_post = true; }% handle_post() %{ if(is_post) deploy(content_length); }% Handler SpecString Matcher Primitive Spec Embedded C++ code deploy(var) function to control match state Check vulnerability predicates here

18. 18 Binary Protocols uint16 name_len = bin_matcher IPP.binpac:IPP_Attr_Data.name_len handler handle_name() default; handle_name() %{ if(name_len > 8192) // throw exception }% VESPA VESPA controls: –vulnerability state –predicate evaluation

19. 19 Binary Protocols binpac controls protocol binary traversal uint16 name_len = bin_matcher IPP.binpac:IPP_Attr_Data.name_len handler handle_name() default; handle_name() %{ if(name_len > 8192) // throw exception }% type IPP_Attr_Data = record { name_len: uint16; name: bytestring &length = name_len &transient; value_len: uint16; value: bytestring &length = value_len &transient; }; binpac IPP specification VESPA

20. 20 Modifying binpac for Binary Traversal Optimized binpac dynamic memory usage –Pre-allocate one of each object that could be parsed in one object –Remove STL vector storage for all array elements

21. 21 Modifying binpac for Binary Traversal Optimized binpac dynamic memory usage –Pre-allocate one of each object that could be parsed in one object –Remove STL vector storage for all array elements Use &pointer attribute to specify objects that must be dynamically created –e.g., DNS name pointers…

22. 22 Evaluation Focus on vulnerabilities difficult to match with exploit sigs Tested raw vuln sig matcher/parser performance –Network reassembly and reporting stages studied elsewhere Test System –2.6 GHz AMD Athlon64 –4GB RAM –Ubuntu Linux 2.6.22-x86-64

23. 23 Tested Vulnerabilities HTTP/IPP –Negative Content-Length causes integer overflow –uint16 name_len used to store size of 8KB buffer DNS –Pointer cycle can cause denial of service WMF –Vulnerable feature: allows arbitrary abort procedure to execute malicious code

24. 24 Memory Micro-benchmarks 6x to 40x reduction in number of calls to new IPP and WMF call new 6x for any file DNS proportional to num of DNS pointers Protocolbinpactraversal DNS15,8122,296 IPP1,360432 WMF3,824312 Protocolbinpactraversal DNS53914 IPP336 WMF946 Bytes allocated per message Calls to new/malloc per message

25. 25 Memory Micro-benchmarks 6x to 40x reduction in number of calls to new IPP and WMF call new 6x for any file DNS proportional to num of DNS pointers Protocolbinpactraversal DNS15,8122,296 IPP1,360432 WMF3,824312 Protocolbinpactraversal DNS53914 IPP336 WMF946 Bytes allocated per message Calls to new/malloc per message

26. 26 String Primitive Micro-benchmarks Multi-string matching dominates text performance VESPA approximates performance of pattern based IDS for simple signatures

27. 27 Parser Performance VESPA outperforms binpac by 3 to 5 times

28. 28 Parser Performance VESPA DNS considerably faster than binpac –Recall, hand-coded 9x faster than VESPA (2.6 Gbits/s) –Room for improvement in binary traversal

29. 29 Related Work Pattern Matching –Wu-Manber, Aho-Corasik, flex, pcre, XFA, Protomatching Vulnerability Signatures –Shield, GAPA, binpac, NetShield, Prospector IDS/IPS –Snort, Bro, SafeCard

30. 30 Conclusions Key Insight: Vulnerability signatures often do not require full protocol parsing –Specialize protocol parser to signature matching Developed VESPA language and architecture –3-5 times faster than binpac –Performance tied to speed of primitives Able to hardware accelerate multi-string matching Improved performance of binary traversal Vulnerability signatures can be matched at 1 Gbps+ –Suitable for server NICs, switches, inline IPS

31. 31 Thank you! Questions?