Multi-Core Packet Scattering to Disentangle Performance Bottlenecks Yehuda Afek Tel-Aviv University

Anat Bremler-Barr David HayYotam Harchol Yaron Koral Joint work with This work was supported by European Research Council (ERC) Starting Grant no

Deep Packet Inspection IPS/IDS/FW Heaviest processing part: Search for malicious patterns in the payload 1.Pipeline multi-core, not efficient. – Imbalance of pipeline stations, DPI much heavier 2.Parallel multi-core?

Multi-Core Deep Packet Inspection (DPI) Option 1: Each core a subset of patterns Core 1 Core 2 Core 3 Core 4 Pattern Set 1 Pattern Set 2 Pattern Set 3 Pattern Set 4

Multi-Core Deep Packet Inspection (DPI) Option 1: Each core a subset of patterns Core 1 Core 2 Core 3 Core 4 Pattern Set 1 Pattern Set 2 Pattern Set 3 Pattern Set 4

Multi-Core Deep Packet Inspection (DPI) Option 1: Each core a subset of patterns Core 1 Core 2 Core 3 Core 4 Pattern Set 1 Pattern Set 2 Pattern Set 3 Pattern Set 4

Multi-Core Deep Packet Inspection (DPI) Option 1: Each core a subset of patterns Option 2: All cores are the same, Load-balance between cores Core 1 Core 2 Core 3 Core 4 Pattern Set 1 Pattern Set 2 Pattern Set 3 Pattern Set 4

Multi-Core Deep Packet Inspection (DPI) Option 2: All cores are the same, Load-balance between cores Core 1 Core 2 Core 3 Core 4 DPI

Multi-Core Deep Packet Inspection (DPI) Option 2: All cores are the same, Load-balance between cores Core 1 Core 2 Core 3 Core 4 DPI

Complexity DoS Attack Over NIDS Easy to craft – very hard to process packets 2 Steps attack: Attacker Internet 2. Steal CC. 1. Kill IPS/FW

Attack on Security Elements Combined Attack: DDoS on Security Element exposed the network – theft of customers’ information

Attack on Snort The most widely deployed IDS/IPS worldwide. Heavy packets rate

OUR GOAL: A multi-core system architecture, which is robust against complexity DDoS attacks

Airline Desk Example

A flight ticket

Airline Desk Example An isle seat near window!! Three carry handbags !!! Doesn’t like food!!! Can’t find passport!! Overweight!!!

Airline Desk Example

Domain Properties 1.Heavy & Light customers. 2.Easy detection of heavy customers. 3.Moving customers between queues is cheap. 4.Heavy customers have special more efficient processing method. Special training packets

Some packets are much “heavier” than others The Snort-attack experiment Property 1 in Snort Attack

DPI mechanism is a main bottleneck in Snort Allows single step for each input symbol Holds transition for each alphabet symbol Snort uses Aho-Corasick DFA Heavy Packet Fast & Huge Best for normal traffic Exposed to cache-miss attack Best for normal traffic Exposed to cache-miss attack

Crafting HEAVY packets Snort patterns DatabaseMalicious pkts Factory Chop last 2 bytes

Snort-Attack Experiment Cache Main Memory Normal TrafficAttack Scenario Does not require many packets!!!

The General Case: Complexity Attacks Trivial to Craft --- Hard to process packets Domain Properties 1.Heavy & Light packets. 2.Easy detection of heavy packets 3.Moving packets between queues is cheap. 4.Heavy packets have special more efficient processing method.

Property 2 in Snort Attack Detecting heavy packets is feasible

How Do We Detect? May be quickly classified Common states Claim: the general case in complexity attacks!!! threshold Percent non-common states

How Do We Detect? Common States Non Common States Heavy packet : # Not Common States # Common States ≤ α After at least 20 bytes

Domain Properties 1.Heavy & Light packets. 2.Easy detection of heavy packets 3.Moving packets between queues is cheap. 4.Heavy packets have special more efficient processing method.

System Architecture Processor Chip Core #8 NIC Core #1 Q Core #2 Q Q Q Q Detects heavy packets Core #9 Core #10 Routine Mode: Load balance between cores

System Architecture Processor Chip Core #8 Dedicated Core #9 NIC Core #1 Q Core #2 Q Q Q B Dedicated Core #10 B Q Detects heavy packets Alert Mode: Dedicated cores for heavy packets Others detect and move heavy to Dedicated. BB

Inter-Thread Communication Heavy packets queues are non-blocking – no locking mechanisms are used Writers write to the queue in cycles (incrementing every full round) – Cycle ID is stored before and after the passed message Readers read in the opposite direction: – If left ID != right ID then writer is now writing – wait and retry – If left ID > expected ID  overflow

Inter-Thread Communication Non-blocking IN-queues – Only one thread accesses Dedicated queues blocking (using test&set locks) – Non-dedicated threads “steal” packets from the HoL when sending a heavy packet Processor Chip Core #8 Dedicated Core #9 NIC Core #1 Q Core #2 Q Q Q B Dedicated Core #10 B Q BB

Domain Properties 1.Heavy & Light packets. 2.Easy detection of heavy packets 3.Moving packets between queues is cheap. 4.Heavy packets have special more efficient processing method.

Snort uses Aho-Corasick DFA

Full Matrix vs. Compressed Heavy packets rate

Domain Properties 1.Heavy & Light packets. 2.Easy detection of heavy packets 3.Moving packets between queues is cheap. 4.Heavy packets have special more efficient processing method.

Experimental Results

System Throughput Over Time Reaction time can be smaller

Different Algorithms Goodput

Additional Application for MCA 2 The Hybrid-FA-attack experiment

Hybrid-FA Space-efficient data structure for regular expression matching Faster than NFA Structure: – Head DFA – Border states – Tail DFAs More than one state can be active at the same time! s0s0 s7s7 s 12 s1s1 s2s2 s3s3 s5s5 s4s4 C C E D B ED s 14 s 13 s6s6 D s8s8 B s9s9 C s 10 A s 11 B A A.* [^\n]*

Hybrid-FA Attack Normal TrafficAttack Scenario Again: Does not require many packets!!! s0s0 s7s7 s 12 s1s1 s2s2 s3s3 s5s5 s4s4 C C E D B ED s 14 s 13 s6s6 D s8s8 B s9s9 C s 10 A s 11 B A A.* [^\n]* s0s0 s7s7 s8s8 s9s9 s 10 s 11 s 12 s2s2 s5s5 s 13 Input: CDBBCAB

Heavy Packet Detection threshold

MCA 2 With Hybrid-FA

Concluding Remarks A multi-core system architecture Robustness against complexity DDoS attacks In this talk we focused on specific NIDS and complexity attack – MCA 2 can handle more NIDS complexity attacks, like the Bro Lazy-FA We believe this approach can be generalized (outside the scope of NIDS)

Thank You!! Deep packet inspection