Honeycomb Automated IDS Signature Generation using Honeypots Christian Kreibich Jon Crowcroft

Motivation  We’d like to characterize suspicious traffic  IDS signatures are a way to do this  How to focus on relevant traffic? (Evil Bit )  Honeypots have no production value  Their traffic is suspicious by definition  Thus: look for patterns in honeypot traffic

Honeycomb  Name? Nice double meaning...

Honeycomb  Name? Nice double meaning...  Combing for patterns in honeypot traffic

Honeycomb’s Architecture

Honeycomb’s Algorithm

Pattern Detection (I)  Stream reassembly:

Pattern Detection (II)  Longest-common-substring (LCS) on pairs of messages: m 1 : fetaramasalatapatata m 2 : insalataramoussaka  Can be done in O(|m 1 | + |m 2 |) using suffix trees  Implemented libstree, generic suffix tree library  No hardcoded protocol-specific knowledge

Pattern Detection (II)  Longest-common-substring (LCS) on pairs of messages: m 1 : fetaramasalatapatata m 2 : insalataramoussaka  Can be done in O(|m 1 | + |m 2 |) using suffix trees  Implemented libstree, generic suffix tree library  No hardcoded protocol-specific knowledge

Pattern Detection (III)  Horizontal detection:  LCS on pairs of messages  each message independent  e.g. (persistent) HTTP

Pattern Detection (IV)  Vertical detection:  concatenates incoming messages  LCS on pairs of strings  for interactive flows and to mask TCP dynamics  e.g. FTP, Telnet,...

Signature Pool  Limited-size queue of current signatures  Relational operators on signatures:  sig 1 = sig 2 : all elements equal  sig 1  sig 2 : elements differ  sig 1  sig 2 : sig 1 contains subset of sig 2 ’s facts  sig new = sig pool : sig new ignored  sig new  sig pool : sig new added  sig new  sig pool : sig new added  sig pool  sig new : sig new augments sig pool  Aggregation on destination ports

Results  We ran Honeycomb on an unfiltered cable modem connection for three days  Honeyd setup:  fake FTP, Telnet, SMTP, HTTP services, all Perl/Shell scripts.  Other ports: traffic sinks  Some statistics:  649 TCP connections, 123 UDP connections  Full traffic volume: ~1MB  approx. 30 signatures created  No wide-range portscanning

TCP Connections HTTP Kuang2 Virus/Trojan SMB NetBIOS Microsoft SQL Server

UDP Connections NetBIOS Messenger Service Slammer

Signatures created: Slammer  Honeyd log:  :26: udp(17) S :27: udp(17) E : :58: udp(17) S :59: udp(17) E : :15: udp(17) S :16: udp(17) E :  Signature:  alert udp any any -> / (msg: "Honeycomb Thu May 8 09h58m "; content: "| DC C9 B0|B|EB 0E |p|AE|B|01|p|AE|B| |h|DC C9 B0|B|B |1|C9 B1 18|P|E2 FD|5| |P|89E5|Qh.dllhel32hkernQhounthickChGetTf|B9|llQh32.dhws2 f|B9|etQhsockf|B9|toQhsend|BE AE|B|8D|E|D4|P|FF 16|P|8D|E|E0|P|8D|E|F0|P|FF 16|P|BE AE|B|8B 1E 8B 03|=U|8B EC|Qt|05 BE 1C 10 AE|B|FF 16 FF D0|1|C9|QQP|81 F B 81 F |Q|8D|E|CC|P|8B|E|C0|P|FF 16|j|11|j|02|j|02 FF D0|P|8D|E|C4|P|8B|E|C0|P|FF C6 09 DB 81 F3|<a|D9 FF 8B|E|B4 8D C1 E C2 C1 E2 08|)|C2 8D D8 89|E|B4|j|10 8D|E|B0|P1|C9|Qf|81 F1|x|01|Q|8D|E|03|P|8B|E|AC|P|FF D6 EB|"; )  Full worm detected

Signatures created: CodeRedII  Hit more than a dozen times  alert tcp /8 any -> /32 80 (msg: "Honeycomb Tue May 6 11h55m "; flags: A; flow: established; content: "GET /default.ida?XXXXXXXXXXXXXXXXXXXXX  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801%u90 90%u6858%ucbd3%u7801%u9090%u9090%u8190%u00c3%u0003%u8b00%u531b%u53ff%u0078%u0000%u00= a HTTP/1.0|0D 0A|Content-type: text/xml|0A|Content-length: 3379 |0D 0A 0D 0A C8 C |`|E CC EB FE|dg|FF|6|00 00|dg|89|&|00 00 E8 DF |h| D 85|\|FE FF FF|P|FF|U|9C 8D 85|\|FE FF FF|P|FF|U|98 8B D|X|FE FF FF FF|U|E4|=| F 94 C1|=| F 94 C5 0A CD 0F B6 C9 89 8D|T|FE FF FF 8B|u|08 81|~0|9A F 84 C C7|F0|9A E8 0A |CodeRedII|00 8B 1C|$|FF|U|D8|f|0B C0 0F 95 85|8|FE FF FF C7 85|P|FE FF FF |j|00 8D 85|P|FE FF FF|P|8D 85|8|FE FF FF|P|8B|E|08 FF|p|08 FF BD|8|FE FF FF 01|thS|FF|U|D4 FF|U|EC 01|E|84|i|BD|T|FE FF FF|,| C7|,| E8 D F7 D0 0F AF C7 89|F4|8D|E|88|Pj|00 FF|u|08 E E9 01 FF FF FF|j|00|j|00 FF|U|F0|P|FF|U|D0|Ou|D2 E8|;| |i|BD|T|FE FF FF 00|\&|05 81 C7 00|\&|05|W|FF|U|E8|j|00|j|16 FF|U|8C|j|FF FF|U|E8 EB F9 8B|F4)E|84|jd|FF|U|E8 8D 85| |FE FF FF 83 F8 0A|s|C3|f|C7 85|p|FF FF FF 02 00|f|C7 85|r|FF FF …  Full worm due to vertical detection – server replies before all signature-relevant packets seen

Signatures detected: others …  alert tcp /32 any -> / ,3128,4588,6588,8080 (msg: "Honeycomb Mon May 5 19h04m "; flags: S; flow: stateless; )

Signatures detected: others …  alert tcp /32 any -> / ,3128,4588,6588,8080 (msg: "Honeycomb Mon May 5 19h04m "; flags: S; flow: stateless; )  alert udp /32 any -> / (msg: "Honeycomb Thu May 8 12h57m "; content: "| |YOUR EXTRA PAYCHEQUE|00 E1 04|x|0C C | |00|#| |#| | Amazing Internet Product Sells Itself!|0D 0A|Resellers Wanted! GO TO )

Signatures detected: others …  alert tcp /32 any -> / ,3128,4588,6588,8080 (msg: "Honeycomb Mon May 5 19h04m "; flags: S; flow: stateless; )  alert udp /32 any -> / (msg: "Honeycomb Thu May 8 12h57m "; content: "| |YOUR EXTRA PAYCHEQUE|00 E1 04|x|0C C | |00|#| |#| | Amazing Internet Product Sells Itself!|0D 0A|Resellers Wanted! GO TO )  alert tcp /32 any -> /32 80 (msg: "Honeycomb Thu May 8 07h27m "; flags: PA; flow: established; content: "GET /scripts/root.exe?/c+dir HTTP/1.0|0D 0A|Host: www|0D 0A|Connnection: close|0D 0A 0D|"; )

Signature Usability  LCS blindly calculates longest substring: alert tcp any any -> / (msg: "Honeycomb Fri Jul 18 02h40m "; flags: PA; flow: established; content: "| FF|SMBr| |S|C FF FE |b|00 02|PC NETWORK PROGRAM 1.0|00 02|LANMAN1.0|00 02|Windows for Workgroups 3.1a|00 02|LM1.2X002|00 02|LANMAN2.1|00 02|NT LM 0.12"; )  Generated signatures not necessary useful for everyday use

Signature Usability (II)  But this “distraction” can be interesting: alert tcp /32 any -> /32 80 (msg: "Honeycomb Sun Nov 9 19h03m "; flags: A; flow: established; content: "F45dYN1pL3zRApBOj2WCKnO2hiH9UgFzTlLwkFg0OPehaFKCk1gYadTVTcrsHbcz5Gd4qg.9 4xMs7cRE0ivx8.GVNN3YK1yCn8AU8WnuJtrcsEyTtwrH2ivX.w5UvBFGTN8y56ISLjiDeCBxj QVfdZGRllRB9jOG5m70m9keYyNsW2g51WiGzsOY2MCkawAoxAMFsh3rwRLVBtqGLGiXsm9SIr sEF23jQ6nbJM3knX6AbQqfqMBEMxApEgnWqK4xq0ZmmRaWj84uNmyTD3ZBg1KUkXUaAlBEntz hFJIhpWfDaWefyBBf4WsBFzfCO.YFBHIzam2N9GrJhwSHc7vowkdGXXWuvdpqHJowhbLG6KvH ZVjoFkUXqwOaTTK22z0osT9cAR.mRBXmrtCwe5wViX9EWaGHgocWqviXkBbvYZuns5IrXQv28 kBDm4oMoWl7JLvzZ-Wd-18qj.jztV mDPNc0FHsv2N4U4qczZzBssfp6S.8W0Azj9R1wLkjpP Xjr9r8ZOmE7Jyq1-MET-2gW9ETIe tlqd39CjftUnszxCDDAZnsXZeuT1C3xDwefCHI344MF K45Fi4GrZRKHJWUkJkKW622tnCAqR3zRF.MxBrkNcfeVcDkv2fOE0PF8AUCfiewxcA4x1mu3n iSnlx1T-hRcb0l1Q983X8ANPFI8H4vM-TQ vhMkHsvN0nxsUrh9xBm.YZL6Nc300YNle4DGK FNz.8HIQ9ID8mRIGJSGzcPHaq7EXAo67nnkHWw58d4udtwsbrr7NN48v5zjKtBlpklHTTcqjY sKsVWDhqEzDqFMrplBvgHfnjtKUIsBQsLIKgEAu9vXH5tWu3ef4nPT.7Tz9i8pb3DyZBMyqAf 6TkYG5z.UUeZP5BrTTc2XFOY1xfRieOzb.5qgE1GyXMojMNWZqTuZKMWVzW8ZMNXx3ARaxpNC D-LB8oWxCtruMqb-mOuxR2NkMfZMFnLsIouUzQtGZ8RsY2NJEz."; )

Summary  System detects patterns in network traffic  Using honeypot traffic, the system creates useful signatures  Good at worm detection  Todo list  Ability to control LCS algorithm (whitelisting?)  Tests with higher traffic volume  Experiment with approximate matching  Better signature reporting scheme

Thanks!  Shoutouts: a13x hØ  No machines were harmed or compromised in the making of this presentation. 