Design and Implementation of a Consolidated Middlebox Architecture 1 Vyas SekarSylvia RatnasamyMichael ReiterNorbert Egi Guangyu Shi

Need for Network Evolution 2 New devices New applications Evolving threats Policy constraints Performance, Security, Compliance

3 Type of applianceNumber Firewalls166 NIDS127 Media gateways110 Load balancers67 Proxies66 VPN gateways45 WAN Optimizers44 Voice gateways11 Total Middleboxes636 Total routers~900 Network Evolution today: Middleboxes! Data from a large enterprise: >80K users across tens of sites Just network security $10 billion

4 Specialized boxes Narrow interfaces “Point” solutions! Increases capital expenses & sprawl Increases operating expenses Limits extensibility and flexibility Management Key “pain points” 

Motivation High-level idea: Consolidation System design Implementation and Evaluation 5 Outline

Network-wide Controller Key idea: Consolidation 2. Consolidate Management 1. Consolidate Platform Two levels corresponding to two sources of inefficiency 6

Consolidation at Platform-Level 7 ProxyFirewallIDS/IPS AppFilter Today: Independent, specialized boxes Decouple Hardware and Software Commodity hardware: e.g., PacketShader, RouteBricks, ServerSwitch, SwitchBlade Consolidation reduces capital expenses and sprawl

Consolidation reduces CapEx 8 Multiplexing benefit = Max_of_TotalUtilization / Sum_of_MaxUtilizations

Consolidation Enables Extensibility 9 Session Management Protocol Parsers VPN Web Mail IDS Proxy Firewall Contribution of reusable modules: 30 – 80 %

Management consolidation enables flexible resource allocation 10 N1 N3 N2 P: N1  N3 Process (0.4 P) Today: All processing at logical “ingress” Overload! Network-wide distribution reduces load imbalance Process (0.3 P) Process (P)

Motivation High-level idea: Consolidation CoMb: System design Implementation and Evaluation 11 Outline

Network-wide Controller CoMb System Overview 12 General-purpose hardware: e.g., PacketShader, RouteBricks, ServerSwitch, Logically centralized e.g., NOX, 4D Existing work: simple, homogeneous routing-like workload Middleboxes: complex, heterogeneous, new opportunities

Network-wide Controller Processing responsibilities CoMb Management Layer Policy Constraints Resource Requirements Routing, Traffic Goal: Balance load across network. Leverage multiplexing, reuse, distribution 13

Capturing Reuse with HyperApps 14 IDSProxy common Footprint on resource HTTP UDP HTTP NFS Memory CPU HTTP = IDS & Proxy 43 Memory UDP = IDS NFS = Proxy CPU Memory CPU HyperApp: find the union of apps to run Policy, dependency are implicit Need per-packet policy, reuse dependencies! HTTP: 1+2 unit of CPU 1+3 units of mem

Modeling Processing Coverage 15 HTTP N1  N3 N1N2 N3 What fraction of traffic of class HTTP from N1  N3 should each node process? IDS < Proxy 0.4 HTTP: Run IDS < Proxy IDS < Proxy 0.3 IDS < Proxy 0.3

Network-wide Optimization 16 Minimize Maximum Load, Subject to Processing coverage for each class of traffic  Fraction of processed traffic adds up to 1 Load on each node  sum over HyperApp responsibilities per-path A simple, tractable linear program Very close (< 0.1%) to theoretical optimal No explicit Dependency Policy

Network-wide Controller CoMb System Overview 17 General-purpose hardware: e.g., PacketShader, RouteBricks, ServerSwitch, Logically centralized e.g., NOX, 4D Existing work: simple, homogeneous routing-like workload Middleboxes: complex, heterogeneous, new opportunities

CoMb Platform 18 NIC Policy Shim (Pshim) Core1 Core4 IDS … … Proxy Traffic Applications Policy Enforcer Classification: HTTP IDS < Proxy Challenges: Performance Parallelize Isolation Challenges: Lightweight Parallelize Challenges: No contention Fast classification

Parallelizing Application Instances 19 M1 M2 PShim App-per-core Core3 M3 PShim Core1 Core2 HyperApp-per-core M2M3 PShim M1M2 PShim Core1 Core2 - Inter-core communication - More work for PShim + No in-core context switch + Keeps structures core-local + Better for reuse - But incurs context-switch - Need replicas ✔ HyperApp-per-core is better or comparable Contention does not seem to matter!

CoMb Platform Design 20 Hyper App1 Hyper App2 Hyper App4 Hyper App3 Hyper App3 PShim M1M4M1M4M2M3 Q1 Q3 Q2Q4Q5 M1M5 Core 1Core 3Core 2 NIC hardware Contention-free network I/O Core-local processing Parallel, core-local Workload balancing

Motivation High-level idea: Consolidation System design: Making Consolidation Practical Implementation and Evaluation 21 Outline

Implementation 22 Network-wide Management Session Protocol Extensible apps Standalone apps Policy Shim using CPLEX Kernel mode Click Ported logic From Bro  Click Memory mapped Or Virtual interfaces 8-core Intel Xeon with Intel NIC

Consolidation is Practical Low overhead for existing applications Controller takes < 1.6s for 52-node topology 5x better than VM-based consolidation 23

Benefits: Reduction in Maximum Load 24 Consolidation reduces maximum load by X MaxLoad Today /MaxLoad Consolidated

Benefits: Reduction in Provisioning Cost 25 Consolidation reduces provisioning cost X Provisioning Today /Provisioning Consolidated

Discussion Changes traditional vendor business – Already happening (e.g., “virtual appliances”) – Benefits imply someone will do it! – May already have extensible stacks internally! Isolation – Current: rely on process-level isolation – Get reuse-despite-isolation? 26

Network evolution occurs via middleboxes Today: Narrow “point” solutions – High CapEx, OpEx, and device sprawl – Inflexible, difficult to extend Our proposal: Consolidated architecture – Reduces CapEx, OpEx, and device sprawl – Extensible, general-purpose More opportunities – Isolation – APIs (H/W—Apps, Management—Apps, App Stack) 27 Conclusions