Supercharged PlanetLab Platform, Control Overview Fred Kuhns fredk@arl.wustl.edu Applied Research Laboratory Washington University in St. Louis

Fast Path/Slow Path Architecture Remote Login Interface Application Manager out-of-band control shared server exception packets & in-band control exception packets & in-band control Control Interface Filters Parse Lookup Hdr Format output interfaces input interfaces ... ... ... Queue Manager Fast Path

Prototype Organization GPE NPE LC RTM Switch Lookup (2 ME) ExtRx IntTx Queue Manager Key Extract Hdr Format (1 ME) IntRx ExtTx Rate Monitor TCAM DRAM SRAM external interface switch interface ingress side egress side Lookup (1 ME) Rx Tx Queue Manager (2 ME) Key Extract Hdr Format TCAM DRAM SRAM One NP blade (with RTM) implements Line Card separate ingress/egress pipelines Second NP hosts multiple slice fast-paths multiple static code options for diverse slices configurable filters and queues GPEs run standard Planetlab OS with vServers

Development System Network Processor blades General purpose blades dual IXP 2850 NPs 3xRDRAM, 3xSRAM, TCAM dual 10GE interfaces 10x1GE IO interfaces General purpose blades dual Xeons, 4xGigE, disk 10 Gb/s Ethernet switch VLANs for traffic isolation

System Block Diagram … … … External Interfaces SPP Node NPE NPE GPE RTM RTM 10 x 1GbE NPE NPE GPE GPE LC LC PCI xscale PLab OS xscale xscale xscale … … … NPU-A TCAM NPU-B 2x1GE 2x1GE NPU-A NPU-B interface X X Fabric Ethernet Switch (10Gbps, data path) Base Ethernet Switch (1Gbps, control) I2C (IPMI) Control Processor (CP) “Extern” GPEs myPLC* Global Node manager (GNM) Resource DB user login info Shelf manager Global Resource Manager (GRM)

Default Traffic Configurations external interface to fabric and base (additional GPEs) Control messages sent over an isolated base Ethernet switch. For isolation and security PE NPE GPE LNM Line card performs NAT like function for traffic from vservers. … MP LRM root context planetlab OS 4 3 2 1 x x x x 10GbE (fabric, data) 5 6 1GbE (base, control) x x Substrate LC mux CP GRM user login info GNM Resource DB PLC Default: traffic forwarded to CP over 10Gbps Ethernet switch (aka fabric) myPLC*

Logging Into a Slice … LNM Host LRM MP PE NPE GPE LNM Host (located within node) … MP LRM root context planetlab OS Once authenticated, session forwarded to appropriate GPE and vserver. 4 3 2 1 x x x x 10GbE (fabric, data) 5 6 1GbE (base, control) x x Substrate LC mux CP GRM ssh fwder user login info GNM Resource DB ssh connection directed to CP for user authentication PLC myPLC*

Update Local Slice Definitions PE NPE GPE LNM Host (located within node) … MP LRM root context planetlab OS 4 3 2 1 x x x x 10GbE (fabric, data) 5 6 1GbE (base, control) x x Substrate update local database, allocate slice instances (slivers) to GPE nodes LC mux CP GRM user login info GNM Resource DB retrieve/update slice descriptions PLC myPLC* slices ... slices ... slices ... slices ...

Creating Local Slice Instance create new slice retrieve/update slice descriptions slices ... PE NPE GPE LNM Host (located within node) … MP LRM root context planetlab OS 4 3 2 1 x x x x 10GbE (fabric, data) 5 6 1GbE (base, control) x x Substrate LC mux CP GRM user login info GNM Resource DB PLC myPLC* slices ... slices ... slices ...

Allocating NPE (Creating Meta-Router) Allocate NPE sliver {code option, SRAM, Interfaces/Ports, etc} Open local socket for exception and local delivery traffic; return to client vserver NPE FP - fast path lkup tbl Control Interface Fast Path ... SRAM TCAM PE GPE LNM Host (located within node) … FPk MP LRM root context planetlab OS Forward request to global resource manager Returns status and assigned global Port number 4 3 2 1 x VLANk x x x 10GbE (fabric, data) 5 6 1GbE (base, control) x x Substrate LC mux CP MI1 GRM user login info GNM Resource DB Allocate shared NPE resources, associate with new slice fast path {SRAM block; # filter table entries; # of queues queues; # of packet buffers; code option; amount of SRAM required; total reserved bandwidth} Allocate global UDP port for requested interface(s); configure Line card. PLC myPLC* Allocate and Enable VLAN to isolate internal slice traffic, VLANk

Managing the Data Path Allocate or Delete NPE Slice instance Add, remove or alter filters each slice is allocated a portion of the NPE’s TCAM Read or write to per slice memory blocks in SRAM each slice is allocated a block of SRAM Read counters one time or periodic Set Queue rate or threshold. Get queue lengths NPE SRAM TCAM GPE LNM DPl DPl FPk MP LRM root context planetlab OS 2 1 x x 10GbE (fabric, data) 6 1GbE (base, control) x CP GRM user login info GNM Resource DB myPLC* FP - fast path