Design of a High Performance PlanetLab Node John DeHart jdd@arl.wustl.edu http://www.arl.wustl.edu/arl
Revision History (10/06 - ??/??) 10/3/06 (JDD) Added info on buffer descriptor 10/11/06 (JDD) Corrections to order of Src and Dst IP Addr in IP packet headers 10/12/06 (JDD) Add proposed change to Lookup input and output data formats 10/24/06 (JDD) Updates to Egress Lookup Key 10/26/06 (JDD) Added HW definitions 11/02/06 (JDD) Added diagram of 2 code options, 4 slices, 4 interfaces per slice. 11/15/06 (JDD) Adding November Intel Demo config info
Revision History (6/06 – 9/06) 6/27/06 (JDD): Created from Diversified Router version 7/5/06 (JDD): Still in progress… 7/6/06 (JDD): 7/10/06 (JDD) Minor modifications based on comments from Jing and Brandon. 7/17/06 (JDD) Updated Data between Key_Extract, Lookup and Hdr_format on LC projects to consistently use the same format for the second word of data: IP Pkt Length (16b), Reserved(8b), Eth Hdr Len(8b) 8/30/06 (JDD) Changed fields in hdr_format to QM for IPv4 MR to make the agree with LC projects. 9/8/06 (JDD) Modified IPv4 Memory Map Clean separation between Init memory blocks and Dynamic memory blocks Modified LC Egress Lookup Key (add Sport) and Result (add counter index) 9/12/06 (JDD and BDH) Add Slice Memory Ptr to NN ring structures. Add Code option to be passed through to Header Format Update Egress Data formats for Key Extract, Lookup and Hdr Format
Overview These slides are still a bit rough, but it should get us started Three Project Goals PlanetLab Node ONL Router Diversified Router First Priority: PlanetLab Node for November Demo Phase 0.0: external GE Switch, 1 GPE, 1 LC, >=1 NPE Phase 0.5: same but with Switch Blade What is impact on Rx, Tx, QM? Phase 1: same but with Multiple GPEs (each as its own PlanetLab Node) Phase 2: same but with Multiple GPEs, unified as one PlanetLab Node. Heavy emphasis in these slides is on Phase 0.0 Probable hardware configuration: 1 NP Blade for LC 1 NP Blade for NPE 1 GP Blade External GE Switch Possible additions: 1 Switch Blade in place of external switch 1 NP Blade for a second NPE
Overview Assume we will use VLANs internally to isolate and identify MRs. Assume we can set Ethernet MAC addresses on blades 40 bits fixed, 8 bits variable View PlanetLab UDP/IP as a new Substrate Link Type No Substrate Headers used internally or externally All packets/frames are IP packets in Ethernet Frames External LC GPE, GPE LC LC NPE, NPE LC NPE GPE, GPE NPE In effect, UDP/IP Headers are the Substrate Header. MN Internal Header still used between MPEs (NPE GPE) Static Shared NP implementation of MRs. Limited, predefined MR code options Statically loaded Different MR slices run the same code, just use different Filters in Lookup table. Queuing is slightly different on each system (LC_Ingress, LC_Egress, NPE) What about ARP? Do we need/want to support it for November? We’ll need it eventually.
System View … exception packets use internal port numbers slice pkt GPE exception packets use internal port numbers VS VS NPE … slice pkt IPH MN-H Kernel/VNET Switch LC Default filter directs packet to GPE Filter directs packet to NPE IPH IPH daddr= nextNode daddr= thisNode IPH IPH daddr= nextNode slice pkt slice pkt slice pkt slice pkt daddr= thisNode
System View: External Switch PLC GPE Switch / 5 1Gb / 1 1Gb Net R T M LC / 1 1Gb R T M NPUA / 5 1Gb 4 1Gb / Local Host x4 NPUB / 5 1Gb Phase 0.0: External GE Switch (16 Ports connected) LC RTM: 1 GE Interface connected to PLC/myPLC via Network 4 GE Interfaces connected directly to Local Hosts Extra data sources and sinks NP RTM 5 GE Interfaces used by NPUA 5 GE Interfaces used by NPUB GPE 1 Front panel GE Interface
System View: External Switch PLC GPE Switch / 5 1Gb / 1 1Gb Net R T M LC / 1 1Gb R T M NPUA / 5 1Gb 4 1Gb / Local Host x4 NPUB / 5 1Gb TP GigE Fiber GigE (max of 4 slots on switch)
System View: Switch Blade PLC GPE Switch Blade / 1 1Gb Net R T M LC / 1 1Gb / 1 10Gb NPE-1 / 1 10Gb Local Host x9 / 9 1Gb Phase 05: Switch Blade LC RTM 1GE Interface connected to PLC/myPLC via Network 9 GE Interfaces connected directly to Local Hosts 1 10Gb interface via backplane NPE Will not need an RTM GPE 1 GE Interface on the Fabric connector
PlanetLab Ingress LC Input Frame New PlanetLab Substrate Link Type: Configured SL Type LC is told at boot/init time that this is its one and only SL Type. Similar to the way P2P-DC is handled. SL Type: 0101b Port: May be a don’t care IP DAddr: Verifies that packet is for our node IP Proto = UDP Could be a UDP tunnel to a slice UDP DPort: Indicates which slice Default route is to the GPE Key = SL=0101b Port: May be a don’t care. IP DAddr = our node address DstAddr (6B) DstAddr (6B) SrcAddr (6B) Ethernet Header SrcAddr (6B) Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) ID/Flags/FragOff (4B) TTL (1B) TTL (1B) Protocol = UDP (1B) Protocol = UDP (1B) Hdr Cksum (2B) Hdr Cksum (2B) Src Addr (4B) Src Addr (4B) Header IP Dst Addr (4B) Dst Addr (4B) IP Options (0-40B) IP Options (0-40B) Src Port (2B) Src Port (2B) UDP Header Dst Port (2B) Dst Port (2B) UDP length (2B) UDP length (2B) UDP checksum (2B) UDP checksum (2B) UDP Payload (MN Packet) UDP Payload (MN Packet) PAD (nB) PAD (nB) Ethernet Trailer CRC (4B) CRC (4B) PlanetLab IPv4 Key(0x5) (64 bits) SL(4b) 0101 Port (4b) IP DAddr (32b) IP Proto (8b) UDP DPort (16b)
PlanetLab Ingress LC Processing Ingress Processing: Portions are similar to IPv4 MR Parse. IP Header checks/validation: Check that version is IPv4 Check IP Header checksum Ignore options (Leave as is and forward on) Drop if fragmented (What if GPE bound?) Extract IP Protocol IP Dst Addr UDP Dst Port Or whatever is in the 2B that would be the UDP port if the IP Protocol were UDP We shouldn’t have to worry about what might be in this field if the IP Protocol is not UDP Perform Lookup Result contains Per MI Stats/Counter Index Ethernet DAddr for destination blade VLAN for destination slice (if needed) No changes made to IP Header Re-write Ethernet Header Configured with Ethernet SAddr for LC Total number of 8-Byte reads: No VLAN, No IP Options: 4 No VLAN, Max IP Options: 9 VLAN, No IP Options: 5 VLAN, Max IP Options: 14 DstAddr (6B) Ethernet Header DstAddr (6B) SrcAddr (6B) SrcAddr (6B) Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) ID/Flags/FragOff (4B) TTL (1B) TTL (1B) Protocol = UDP (1B) Protocol = UDP (1B) Hdr Cksum (2B) Hdr Cksum (2B) Src Addr (4B) Src Addr (4B) Header IP Dst Addr (4B) Dst Addr (4B) IP Options (0-40B) IP Options (0-40B) Src Port (2B) Src Port (2B) UDP Header Dst Port (2B) Dst Port (2B) UDP length (2B) UDP length (2B) UDP checksum (2B) UDP checksum (2B) UDP Payload (MN Packet) UDP Payload (MN Packet) PAD (nB) PAD (nB) Ethernet Trailer CRC (4B) CRC (4B) Indicates fields that need to be read Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab Ingress LC Output Frame Ethernet Header is only thing that changes: DAddr: MAC Address of GPE/NPE (Result) 40 bits are static 8 bits variable and stored in Result SAddr: MAC Address of LC (static) Type = 802.1Q (static) VLAN = Slice VLAN (Result) Type = IP (static) Total number of 8-Byte Reads: 1 Need to read first part of IP header so when we do the write of last part of ethernet header we can fill out the 8-Byte Write. Total number of 8-Byte Writes: 3 DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) IP Options (0-40B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) PAD (nB) Ethernet Trailer CRC (4B) Indicates fields that need to be written Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab NPE Input Frame from LC Ethernet Header: DstAddr: MAC address of NPE SrcAddr: MAC address of LC VLAN: One VLAN per MR (MR == Slice) IP Header: Dst Addr: IP address of this node (phase 0) Src Addr: IP address of previous hop Protocol: UDP UDP Header: Dst Port: Identifies input tunnel Src Port: with IP Src Addr identifies sending entity DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) IP Options (0-40B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) PAD (nB) Ethernet Trailer CRC (4B) Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab NPE Output Frame to LC Re-writes IP Header Dst Addr: Next Hop Src Addr: NPE’s IP Address Re-writes UDP Header Src Port: NPE’s end of the tunnel to next hop Dst Port: Other end of tunnel to next hop. UDP checksum? Re-writes Ethernet Header DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) IP Options (0-40B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) PAD (nB) Ethernet Trailer CRC (4B) Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab NPE Local Delivery Frame FROM GPE IP Header Dst Addr: NPE’s IP Address Src Addr: GPE’s IP Address No IP Options UDP Header Src Port: GPE’s end of the tunnel Dst Port: NPE’s end of the tunnel UDP checksum? MN Internal Header As defined for diversified router DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) MN Internal Hdr (4-10B) PAD (nB) CRC (4B) Ethernet Trailer Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab NPE Local Delivery Frame TO GPE Re-writes IP Header Dst Addr: GPE’s IP Address Src Addr: NPE’s IP Address No IP Options Re-writes UDP Header Src Port: NPE’s end of the tunnel to GPE Dst Port: GPE’s end of the tunnel from NPE UDP checksum? MN Internal Header Need to look at the details of what needs to go in here now that we have no explicit RxMI and TxMI fields/values. Ethernet Header DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) MN Internal Hdr (4-10B) PAD (nB) CRC (4B) Ethernet Trailer Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab NPE Exception Path Frame FROM GPE IP Header Dst Addr: NPE’s IP Address Src Addr: GPE’s IP Address No IP Options UDP Header Src Port: GPE’s end of the tunnel Dst Port: NPE’s end of the tunnel UDP checksum? MN Internal Header As defined for diversified router DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) MN Internal Hdr (4-10B) PAD (nB) Ethernet Trailer CRC (4B) Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab NPE Exception Path Frame TO GPE Re-writes IP Header Dst Addr: GPE’s IP Address Src Addr: NPE’s IP Address No IP Options Re-writes UDP Header Src Port: NPE’s end of the tunnel to GPE Dst Port: GPE’s end of the tunnel from NPE UDP checksum? MN Internal Header As defined for diversified router DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) MN Internal Hdr (4-10B) PAD (nB) CRC (4B) Ethernet Trailer Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab Egress LC Input Frame Ethernet Header addressed to LC IP Packet should be complete and LC does not need to touch it. (Phase 0) Should not even need to do hdr checksum Lookup Key: IP Protocol (8b) UDP Sport (16b) DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) IP Options (0-40B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) PAD (nB) Ethernet Trailer CRC (4B) Indicates fields that need to be read Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab Egress LC Processing Egress Processing: Extract IP Dst Addr and UDP Sport Perform Lookup Need to generate MAC DAddr for next hop VLAN, maybe Assume there are a limited number of next hops: Local Hosts Routers connected to the local subnet. Result contains Per MI Stats/Counter Index Ethernet DstAddr VLAN if needed OR L2 Lookup Table Index L2 Lookup Table Entry: L2 Header Size (14B or 18B) 18 Bytes of Data No changes made to IP Header Re-write Ethernet Header Configured with Ethernet SAddr for LC Total number of 8-Byte reads: 2 DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) IP Options (0-40B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) PAD (nB) Ethernet Trailer CRC (4B) Indicates fields that need to be read Indicates 8-Byte Boundaries Assuming no IP Options
PlanetLab Egress LC Output Frame Ethernet Header is only thing that changes. DAddr: MAC Address of next hop (Result) SAddr: MAC Address of LC (static) Optional Type = 802.1Q (Result) VLAN = Slice VLAN (Result) Type = IP (static) Total number of 8-Byte Reads: 1 Need to read first 2 or 6 Bytes of the IP header so when we do the write of last part of ethernet header we can fill out the 8-Byte Write. Total number of 8-Byte Writes: 2 or 3 DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Dst Addr (4B) IP Options (0-40B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) PAD (nB) Ethernet Trailer CRC (4B) Indicates fields that need to be written Indicates 8-Byte Boundaries Assuming no IP Options
Queuing Assume we are using an external GE switch NPE Queueing: LC RTM: 5 internal ports go to switch for traffic to/from NPE(s) and GPE(s) 5 external ports 1 to Internet 4 to local hosts NPE RTM 5 ports used by NPUA to switch Each port is associated with 1 of the LC’s external ports 5 ports used by NPUB to switch GPE 1 GE port to switch NPE Queueing: Needs Rate control per port
NPE Queuing Queueing on a per port basis Ports split across NPUA and NPUB (Phase 0.0) Rate control per port Rate control will need to be dynamically adjustable so we can balance bandwidth usage across NPs and GPE Each MR gets N queues per port MRs choose how to use the N queues Quantum assigned per MR for each port MRs can choose how to split among their N queues May or may not make sense to assign queues on a per MI basis Quantum being split across the queues means that an MR with only one active MI may not get its “fair” share.
NPE Queuing N P U A QM/Schd QM/Schd N P U B 5-Port Tx ... Port 1(to LC Port 1) 5-Port Tx R T M SPI Switch Port 2(to LC Port 2) N P U A Port 3(to LC Port 3) Port 4(to LC Port 4) Port 5(to LC Port 5) QM/Schd ... Port 6 (to LC Port 1) N P U B Port 7(to LC Port 2) Port 8(to LC Port 3) Port 9(to LC Port 4) Port 10(to LC Port 5)
LC Ingress Queuing 1 5-port QM support 5 GPEs Each “port” supports one GPE Only one queue needed per port. Rate control, 1 Gb/s per port In Phase 0.0, HW flow control will provide/enforce this rate. 1 5-port QM support all NPEs Each “port” supports one NPE Rate control should not be needed
LC Egress Queuing Queueing on a per MR per port basis Similar to NPE Queuing, but each MR gets 1 queue per port Default is equal quantum assigned to each MR on each port All get equal “fair share” Allow for changing quanta.
LC: Functional Blocks S W I T C H Phy Int Rx (2 ME) Key Extract (2 ME) Lookup (2 ME) Hdr Format (1 ME) QM/Schd (2 ME) Switch Tx (2 ME) S W I T C H Phy Int Tx (2 ME) QM/Schd (2 ME) Hdr Format (1 ME) Lookup (2 ME) Key Extract (1 ME) Switch Rx (2 ME)
LC Ingress: Functional Blocks Phy Int Rx Key Extract Lookup Hdr Format QM/Schd Switch Tx S W I T C H RBUF Buf Handle(32b) Eth. Frame Len (16b) Reserved (12b) Port (4b) Rx (2 Microengines): Function: Coordinate transfer of packets from RBUF to DRAM
LC Ingress: Functional Blocks Phy Int Rx Key Extract Lookup Hdr Format QM/Schd Switch Tx S W I T C H Buf Handle(32b) Buf Handle(32b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) Eth. Frame Len (16b) Reserved (12b) Port (4b) Lookup Key[63-32] (32b) Lookup Key[ 31-0] (32b) Key_Extract (1 Microengine): Function: Extracts lookup key. Peel ARP packets off and send to XScale??? Lookup Key (64b): SL Type (4b): 0101b Port (4b): May not be needed IP DAddr (32b) IP Proto (8b) UDP DPort (16b) Notes: Frame offset in buffer is a constant and does not need to be read from Buffer Descriptor Ethernet Hdr Length should be passed along chain so Hdr Format can figure out where to start writing its stuff. Ethernet Header could have different lengths depending on whether VLANs are present or not. IP Hdr 1st Word (32b)
LC Ingress: Functional Blocks Phy Int Rx Key Extract Lookup Hdr Format QM/Schd Switch Tx S W I T C H Buf Handle(32b) Buf Handle(32b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) Lookup Key[63-32] (32b) VLAN (16b) Stats Index (16b) Lookup Key[ 31-0] (32b) DAddr (8b) Port (4b) QID (20b) IP Hdr 1st Word (32b) IP Hdr 1st Word (32b) Lookup: Notes on next page
LC Ingress: Functional Blocks Lookup: Function: Performs Lookup and passes result on to Hdr Format. Lookup Key (64b): SL Type (4b): 0101b Port (4b): May not be needed IP DAddr (32b) IP Proto (8b) UDP DPort (16b) Lookup Result (56b): DAddr (8b): only 8 bits of Ethernet DAddr are variable, other 40 are static per node. VLAN (12b) QID (20b) Stats Index (16b) Port (4b): For case with external switch it is the actual physical interface to use Also one port per GPE and one port per NPE For case with switch blade, it is just used to spread traffic across QM/Scheduler? Notes: Does Lookup Key need to include Port? Seems like it should not. Does Lookup still need Frame Length? Will it be maintaining any Byte Counters? Result should not have to include RxMI, it is not used for anything. Stats Index may be a Per MI stats index if desired.
LC Ingress: Functional Blocks Phy Int Rx Key Extract Lookup Hdr Format QM/Schd Switch Tx S W I T C H Buf Handle(32b) Buffer Handle(32b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) Rsv (4b) Port (4b) Rsv (4b) QID(20b) VLAN (16b) Stats Index (16b) DAddr (8b) Port (4b) QID (20b) Frame Length (16b) Stats Index (16b) IP Hdr 1st Word (32b) Hdr Format: Function: From lookup result: re-writes just the ethernet header in DRAM to make frame ready to transmit. Extract QID, Port, Stats Index and Frame Length to pass on to QM/Scheduler May need to increment a counter based on Stats Index. Notes: Pass Size on to QM/Scheduler so it does not have to read buffer descriptor for Enqueue to update Q Length. Offset to beginning of old Ethernet header should be constant but we don’t necessarily know how long it was so we don’t know where to put our new one. Ethernet Hdr Len is used to determine where new header should go
LC Ingress: Functional Blocks Phy Int Rx Key Extract Lookup Hdr Format QM/Schd Switch Tx S W I T C H Buffer Handle(32b) Buffer Handle(24b) Rsv (3b) Port (4b) V 1 Rsv (4b) Port (4b) Rsv (4b) QID(20b) V: Valid Bit Frame Length (16b) Stats Index (16b) QM/Scheduler (See Sailesh’s slides for more details) Function: Enqueue and Dequeue from queues Scheduling algorithm Drop Policy Notes:
LC Ingress: Functional Blocks Phy Int Rx Key Extract Lookup Hdr Format QM/Schd Switch Tx S W I T C H Buffer Handle(24b) Rsv (3b) Port (4b) V 1 TBUF V: Valid Bit Switch TX: Function: Coordinate transfer of packets from DRAM to TBUF Notes:
LC Egress: Functional Blocks Phy Int Tx QM/Schd Hdr Format Lookup Key Extract Switch Rx S W I T C H Buf Handle(32b) RBUF Eth. Frame Len (16b) Reserved (12b) Port (4b) Rx: Function: Coordinate transfer of packets from RBUF to DRAM Notes: Do we need port? May not make sense to remove it since, it is there for other versions of Rx.
LC Egress: Functional Blocks Phy Int Tx QM/Schd Hdr Format Lookup Key Extract Switch Rx S W I T C H Buf Handle(32b) Buf Handle(32b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) Eth. Frame Len (16b) Reserved (12b) Port (4b) IP DAddr (32b) Lookup Key IP Proto (8b) Lookup Key – UDP SPort (16b) Reserved (8b) IP Hdr 1st Word (32b) Key_Extract: Function: Extracts lookup key Notes:
LC Egress: Functional Blocks Phy Int Tx QM/Schd Hdr Format Lookup Key Extract Switch Rx S W I T C H Buf Handle(32b) Buf Handle(32b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) IP DAddr (32b) IP DAddr (32b) Lookup Result [63-32] (32b) Lookup Key IP Proto (8b) Lookup Key – UDP SPort (16b) Reserved (8b) Lookup: Function: Performs Lookup and passes result on to Hdr Format. Lookup Key: IP Protocol (8b) UDP Sport (16b) Lookup Result (52b): VLAN (12b): Value of 0x000 or 0xFFF, indicates invalid? QID (20b) Port (4b) Stats/Counter Index (16b) Static values for Egress Ethernet address: Ethernet SAddr Types: IP and/or 802.1Q Notes: Lookup does no processing on the lookup result. Lookup Result [31-0] (32b) IP Hdr 1st Word (32b) IP Hdr 1st Word (32b)
Stats/Counter Index (16b) Lookup Result Buf Handle(32b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) IP DAddr (32b) Rsvd (4b) VLAN(12b) Stats/Counter Index (16b) Rsvd (4b) Port (4b) Rsvd (4b) QID (20b) IP Hdr 1st Word (32b)
LC Egress: Functional Blocks Phy Int Tx QM/Schd Hdr Format Lookup Key Extract Switch Rx S W I T C H Buffer Handle(32b) Buf Handle(32b) IP Pkt Length (16b) Reserved (8b) Eth Hdr Len (8b) Rsv (4b) Port (4b) Rsv (4b) QID(20b) IP DAddr (32b) Ethernet Frame Length (16b) Cntr Index (16b) Lookup Result [63-32] (32b) Lookup Result [31-0] (32b) IP Hdr 1st Word (32b) Hdr Format: Function: From lookup result: re-writes ethernet header in DRAM to make frame ready to transmit. Extract QID and frame length to pass on to QM/Scheduler Notes: Pass Size on to QM/Scheduler so it does not have to read buffer descriptor for Enqueue to update Q Length.
LC Egress: Functional Blocks Phy Int Tx QM/Schd Hdr Format Lookup Key Extract Switch Rx S W I T C H Buffer Handle(32b) Buffer Handle(24b) Rsv (3b) Port (4b) V 1 Rsv (4b) Port (4b) Rsv (4b) QID(20b) V: Valid Bit Ethernet Frame Length (16b) Cntr Index (16b) QM/Scheduler (See Sailesh’s slides for more details) Function: Enqueue and Dequeue from queues Scheduling algorithm Drop Policy Memory Accesses: DRAM: None SRAM: Q-Array Reads and Writes Scheduling Data Structure Reads and Writes QLength Data Structure Reads and Writes Dequeue: Read Buffer Descriptor to retrieve Packet Size Buffer Descriptor Accesses: Read packet size Notes:
LC Egress: Functional Blocks Phy Int Tx QM/Schd Hdr Format Lookup Key Extract Switch Rx S W I T C H TBUF Buffer Handle(24b) Rsv (3b) Port (4b) V 1 V: Valid Bit Switch TX: Function: Coordinate transfer of packets from DRAM to TBUF Memory Accesses: SRAM: Read Buffer Descriptor DRAM: Transfer to TBUF Buffer Descriptor Accesses: Read Size and Offset Notes: Calculate DRAM address based on SRAM Descriptor address in buffer handle
NPE Functional Blocks Phase 0.0 Each NPU will only support 5 LC Interfaces. Only need 1 Tx ME Only need 1 QM ME Lookup (1-2 ME) Rx (2 ME) 5-Port Tx (1 ME) QM/Schd Demux Hdr Format (1-3 ME) Parse
NPE Functional Blocks Later Phases We may need some input queuing/buffering to absorb bursts at high input rates (10Gb/s, 20MPkts/s) to keep Parse from being overloaded. Add an SRAM Ring between Demux and Parse instead of NN Ring If needed, add an extra ME to read from SRAM ring and put into NN ring Lookup (1-2 ME) Rx (2 ME) 5-Port Tx (1 ME) QM/Schd Demux Hdr Format Parse
IPv4 MR Functional Blocks Lookup Rx Tx QM Parse Header Format DeMux RBUF Buf Handle(32b) Port (8b) Reserved Eth. Frame Len (16b) Rx: Function Coordinate transfer of packets from RBUF to DRAM Notes: We’ll pass the Buffer Handle which contains the SRAM address of the buffer descriptor. From the SRAM address of the descriptor we can calculate the DRAM address of the buffer data.
IPv4 MR Functional Blocks Lookup Rx Tx QM Parse Header Format DeMux Buf Handle(32b) Port (8b) Reserved Eth. Frame Len (16b) Rx UDP DPort (16b) Buf Handle(32b) Slice ID (VLAN) (16b) MN Frm Offset (16b) MN Frm Length(16b) Rx IP SAddr (32b) Reserved (12b) Rx UDP SPort (16b) Code (4b) DstAddr (6B) SrcAddr (6B) Ethernet Header Type=802.1Q (2B) VLAN (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol = UDP (1B) Hdr Cksum (2B) Src Addr (4B) Header IP Slice Data Ptr (32b) Dst Addr (4B) SRAM Pointer/Range to static data area need to be added. IP Options (0-40B) Src Port (2B) UDP Header Dst Port (2B) UDP length (2B) UDP checksum (2B) UDP Payload (MN Packet) Demux notes on next slide PAD (nB) Ethernet Trailer CRC (4B)
Demux Function Notes: Read Pkt Header from DRAM Can we assume IP Header checksum is ok? This frame comes from LC or from GPE. Are they each trusted? Except for the possibility of options, we have to read all parts of the IP header anyway, so maybe go ahead and calculate the checksum if we have the cycles. Extract the following fields from Pkt Header and pass to Parse VLAN: equivalent to Slice ID (Slice == MR) UDP DPort: equivalent to Substrate MI IP SAddr, UDP Sport: extra interface information for planetlab slices. UDP length: equivalent to MN frame length Calculate offset into buffer of start of MN Pkt Header and pass to Parse Based on Slice ID, determine code option. How do we intended to do this? Load a table in Local memory from SRAM at init time? Notes: Code: Identifies “Code Option” Which set of static Parse code to execute. We will allow a very small number of code options. How does Demux identify code option? Port should not be important in this implementation, right? 8-Byte Dram Reads: 5-10 depending on amount of IP Options and whether we need to do IP Hdr Checksum
IPv4 MR Functional Blocks Lookup Rx Tx QM Parse Header Format DeMux Buf Handle(32b) Lookup Key[111-80] DA (32b) Buf Handle(32b) IP Pkt Length (16b) IP Pkt Offset (16b) Lookup Key[ 79-48] SA (32b) Lookup Key[ 47-16] Ports (32b) Lookup Key Proto/TCP_Flags [15- 0] (16b) Exception Bits (12b) Lookup Key[143-112] Slice ID/Rx UDP DPort (32b) L Flags (4b) MN Frm Length(16b) MN Frm Offset (16b) Slice ID (VLAN) (16b) Rx UDP DPort (16b) Rx IP SAddr (32b) Rx UDP SPort (16b) Reserved (12b) Code (4b) Slice Data Ptr (32b) Slice Data Ptr (32b) Reserved (28b) Code (4b) Parse notes on following page
Parse Notes Parse Notes: Function MR-specific header processing Handles IPv4 header validation Decrements TTL and recalculates Hdr Checksum. Generate MR-specific lookup key (144 bits) from packet Generate Exception bits to be passed on to Hdr Format (via Lookup) so Hdr Format can create shim fields for slow path packets going to Control Processor. Input data to Parse is MN Frame Offset and Length Output data from Parse is IP Pkt Offset and Length If there is a MN Internal Header, Parse basically consumes it. Hdr Format needs to start at the beginning of the IP Header and re-write headers upward in the Buffer. If Parse receives a !Reclassify frame/pkt from the CP, it should not decrement the TTL. Assume this was done on the first trip through the MR. it should not generate exceptions again. Code: Identifies “Code Option” Which set of static Parse code to execute. We will allow a very small number of code options. L Flags: bit 0: 0: Normal, 1: Substrate Lookup Need to revisit the details here of what is in the MN Internal Header bit 1: 0: Normal, 1: NH MN Address present in Key Word[1] Key Word[0] = MR/MI Bit 1 should never be set without bit 0 also being set. Notes: UDP DPort (Rx MI) needs to be passed to Header Format (through Lookup) so that Header Format can include it in the MN Internal Header of packets that end up on the exception or Local delivery path. This will let the Control Processor know what interface the exception packets arrived on.
IPv4 MR Functional Blocks Rx DeMux Parse Lookup Header Format QM Tx Lookup Key[111-80] DA (32b) Buf Handle(32b) IP Pkt Length (16b) IP Pkt Offset (16b) Lookup Key[ 79-48] SA (32b) Lookup Key[ 47-16] Ports (32b) Lookup Key Proto/TCP_Flags [15- 0] (16b) Exception Bits (12b) Lookup Key[143-112] Slice ID/Rx UDP DPort (32b) L Flags (4b) Buf Handle(32b) IP Pkt Length (16b) IP Pkt Offset (16b) Slice ID (VLAN) (16b) Rx UDP DPort(16b) R S V d (1b) H (1b) L D (1b) D (1b) Exception Bits (12b) Cntr Index (16b) Tx IP DAddr (32b) Tx UDP DPort (16b) Tx UDP SPort(16b) DA(8b) Port (4b) QID(20b) Slice Data Ptr (32b) Slice Data Ptr (32b) Reserved (28b) Code (4b) Reserved (28b) Code (4b) Lookup notes on next page
IPv4 MR Functional Blocks Rx DeMux Parse Lookup Header Format QM Tx Buf Handle(32b) Buf Handle(32b) IP Pkt Length (16b) IP Pkt Offset (16b) IP Pkt Length (16b) IP Pkt Offset (16b) Lookup Key[143-112] Slice ID/Rx UDP DPort (32b) Slice ID (VLAN) (16b) Rx UDP DPort(16b) Lookup Key[111-80] DA (32b) R S V d (1b) H (1b) S R V d (1b) D (1b) L D (1b) Reserved (11b) Cntr Index (16b) Lookup Key[ 79-48] SA (32b) Lookup Key[ 47-16] Ports (32b) Tx IP DAddr (32b) Lookup Key Proto/TCP_Flags [15- 0] (16b) Reserved (16 b) Tx UDP DPort (16b) Tx UDP SPort(16b) DA(8b) Port (4b) QID(20b) Slice Data Ptr (32b) Slice Data Ptr (32b) L Flags (4b) Exception Bits (12b) Reserved (12 b) Code (4b) Reserved (4b) Exception Bits (12b) Reserved (12 b) Code (4b) Lookup notes on next page PROPOSED CHANGE TO LOOKUP INPUT AND OUTPUT DATA FORMATS (10/12/06)
Lookup Function Buf Handle(32b) Result: Output Perform lookup in TCAM based on MR Id and lookup key Result: IP DAddr (32b) UDP SPort (16b) UDP DPort (16b) Eth Daddr (8b) : low order 8 bits. Top 40 are pre-defined Port (4b): Phase 0.0 QID (20b) Cntr Index (16b): Used for incrementing Counters Output Buf Handle Exception Bits: For Parse to communicate to Header format info about exception packets Slice ID (VLAN) Rx UDP DPort IP Pkt Length: Length of just the IP Pkt IP Pkt Offset: Offset from start of buffer to the start of IP Pkt header IP/UDP Header fields: IP DAddr, UDP DPort, UDP SPort QID Port: Phase 0.0 Bits: H: Hit D:Drop, NH: NH MN Address present, MAC: MAC Address needed, LD: Local Delivery Rsv (3b): Reserved, MR Bits: For MR-specific usage Port (4b) QID(20b) DA(8b) Tx IP DAddr (32b) Buf Handle(32b) IP Pkt Length (16b) IP Pkt Offset (16b) Cntr Index (16b) R V S d (1b) D H Exception Bits (12b) L Rx UDP DPort(16b) Slice ID (VLAN) (16b) Tx UDP SPort(16b) Tx UDP DPort (16b) Slice Data Ptr (32b) Reserved (28b) Code (4b)
IPv4 MR Functional Blocks Rx DeMux Parse Lookup Header Format QM Tx Port (4b) QID(20b) DA(8b) Tx IP DAddr (32b) Buf Handle(32b) IP Pkt Length (16b) IP Pkt Offset (16b) Cntr Index (16b) S R V d (1b) D H Exception Bits (12b) L Rx UDP DPort(16b) Slice ID (VLAN) (16b) Tx UDP SPort(16b) Tx UDP DPort (16b) Ethernet Frame Length (16b) Buffer Handle(32b) Cntr Index (16b) QID(20b) Rsv (4b) Port Slice Data Ptr (32b) Reserved (28b) Code (4b) Header Format notes on next page.
IPv4 MR Functional Blocks Header Format Function MR specific packet header formatting MR specific Lookup Result processing Drop bit, Hit/Miss bits, NH, MAC, LD, … LD and Exceptions may go to different UDP Ports. Result: Ethernet DAddr of LC (Result) Constant/Static fields needed: IP DAddr of GPE for exception and LD (configured) IP SAddr (configured) UDP Sport and UDP DPort for exception and LD (configured) QID and Port# to use for exception path and LD (configured) IP Proto = UDP (constant) Other IP Header fields: calculated or constants Ethernet SAddr of NPE (configured) VLAN (passed as input) First Ethernet Type = 802.1Q (constant) Second Ethernet Type = IP (constant)
IPv4 MR Functional Blocks Rx DeMux Parse Lookup Header Format QM Tx Ethernet Frame Length (16b) Buffer Handle(32b) Cntr Index (16b) QID(20b) Rsv (4b) Port Buffer Handle(24b) Rsv (3b) Port (4b) V 1 QM Function CRF queue management for Meta Interface queues For performance reasons, QM may actually be implemented as multiple instances Each instance on a separate ME would support a separate set of Meta Interfaces. See next slide for more details…
QM/Scheduler on Multiple MEs Header Format Input Hlpr (1 ME) QM/Schd (1 ME) Tx MR-1 MR-n . . . QM/Schd (1 ME) Tx QID(20b) Ethernet Frame Length (16b) Buffer Handle(32b) Rsv (4b) Cntr Index (16b) Port NN/Scratch Rings NN Ring Buffer Handle(24b) Rsv (3b) Port (4b) V 1 QID(32b): Reserved (8b) QM ID (3b) QID(17b): 1M queues per QM Input Hlpr would use QM ID to select Scratch ring on which to put request. QM/Sched then sends on its output NN/scratch ring to its associated Tx With 64 entries in Q-Array and 16 entries in CAM, max number of QM/Schds is probably 4 (2 bits). We’ll set aside 3 bits to give us flexibility in the future.
IPv4 MR Functional Blocks Rx DeMux Parse Lookup Header Format QM Tx Buffer Handle(24b) Rsv (3b) Port (4b) V 1 TBUF Tx Function Coordinate transfer of packets from DRAM to TBUF
LC: Buffer Descriptor Hopefully we can use the same buffer descriptor for the LC and the CRF Processing Engine. There might be some fields that are used on one and not on the other but that’s ok (MR_ID, TxMI, VLAN not needed on LC) PE Buffer Descriptor: LW0: buffer_next 32 bits Next Buffer Pointer (in a chain of buffers) LW1: offset 16 bits Offset to start of data in bytes LW1: BufferSize 16 bits Length of data in the current buffer in bytes LW2: reserved 8 bits reserved/unused LW2: reserved 4 bits reserved/unused LW2: free_list 4 bits Freelist ID LW2: packet_size 16 bits (Total packet size across multiple buffers) LW3: MR_ID 16 bits Meta Router ID LW3: TxMI 16 bits Transmit Meta Interface LW4: VLAN 16 bits VLAN LW4: reserved 16 bits reserved/unused LW5: reserved 32 bits reserved/unused LW6: reserved 32 bits reserved/unused LW7: packet_next 32 bits pointer to next packet (unused in cell mode) Leave multi-buffer fields there as a template for the dedicated blade implementation of a jumbo-frame MR. Also reduces changes to Rx, Tx, and QM and reduces potential problems. So, far I haven’t found anything extra that we need on LC. VLAN Packet_Next MR_ID TxMI Free_List Packet_Size Buffer_Next Offset Buffer Descriptor Buffer_Size
Intel Buffer Descriptor Buffer_Next (32b) LW0 Buffer_Size (16b) Offset (16b) LW1 Packet_Size (16b) Free_list (4b) Rx_stat (4b) Hdr_Type (8b) LW2 Input_Port (16b) Output_Port (16b) LW3 Next_Hop_ID (16b) Fabric_Port (8b) Reserved (4b) NHID type (4b) LW4 ColorID (4b) Reserved (4b) FlowID (32b) LW5 Class_ID (16b) Reserved (16b) LW6 Packet_Next (32b) LW7
Our Buffer Descriptor Buffer_Next (32b) Buffer_Size (16b) Offset (16b) LW0 Buffer_Size (16b) Offset (16b) LW1 Packet_Size (16b) Free_list 0000 (4b) Reserved (4b) Reserved (8b) LW2 Reserved (16b) Stats Index (16b) LW3 Reserved (16b) Reserved (8b) Reserved (4b) Reserved (4b) LW4 Reserved (4b) Reserved (4b) Reserved (32b) LW5 Reserved (16b) Reserved (16b) LW6 Packet_Next (32b) LW7
Extra The next set of slides are for templates or extra information if needed
Text Slide Template
Image Slide Template
Hardware Definitions Blade Slot 1: Line Card Blade Slot 2: MetaRouter NPUA: Ingress NPUB: Egress Linux Controller: foghorn.arl.wustl.edu Windows Controller: techX01.arl.wustl.edu Blade Slot 2: MetaRouter NPUA: IPv4 MR #1 NPUB: IPv4 MR #2 (optional) Linux Controller: coffee.arl.wustl.edu Windows Controller: techX03.arl.wustl.edu SPI Connections: LC Blade: RTM[0:4] NPUA[0:4] External links to LC Ingress NPUA[5:9] RTM[5:9] LC Ingress to Internal Links RTM[5:9] NPUB[5:9] Internal links to LC Egress NPUB[0:4] RTM[0:4] LC Egress to External Links MR Blade: NPUA[0:4] RTM[0:4] NPUB[0:4] RTM[5:9] Hardwire Connections: LC RTM[0] coffee LC RTM[4] foghorn LC RTM[5] MR RTM[0] LC RTM[6] MR RTM[1]
HW Demo Quick diagram 6 5 7 4 10 1 8 3 9 2 NPUA 10 9 8 7 6 5 4 3 2 1 MR coffee 7 4 10 1 10 9 8 7 6 5 4 3 2 1 LC 8 3 9 2 NPUA – LC Ingress NPUB – LC Egress
Multiple Slices, Code Options and MetaInterfaces UDP Dport VLAN, UDP Dport VLAN, UDP Dport UDP Dport IPv4 MR 0xC100 0xC100 coffee 192.168.81.1 192.168.82.1 192.168.83.1 192.168.84.1 0x001,0xC100 0x001,0xC100 0xC200 Slice 1 (192.168.91.1) Code Option 1 coffee 0x001,0xC101 0x001,0xC101 0xC200 0xC300 0x001,0xC102 0x001,0xC102 0xC300 0xC400 0x001,0xC103 0x001,0xC103 0xC400 0xC101 foghorn 192.168.81.2 192.168.82.2 192.168.83.2 192.168.84.2 0x002,0xC200 0x002,0xC200 0xC101 0xC201 Slice 2 (192.168.92.1) Code Option 1 foghorn 0x002,0xC201 0x002,0xC201 0xC201 0xC301 0x002,0xC202 0x002,0xC202 0xC301 0xC401 0x002,0xC203 0x002,0xC203 0xC401 NPUA – LC Ingress NPUB – LC Egress 0xC102 xxx 192.168.81.3 192.168.82.3 192.168.83.3 192.168.84.3 0x003,0xC300 0x003,0xC300 0xC102 Slice 3 (192.168.93.1) Code Option 2 xxx 0xC202 0x003,0xC301 0x003,0xC301 0xC202 0xC302 0x003,0xC302 0x003,0xC302 0xC302 0xC402 0x003,0xC303 0x003,0xC303 0xC402 0xC103 yyy 192.168.81.4 192.168.82.4 192.168.83.4 192.168.84.4 0x004,0xC400 0x004,0xC400 0xC103 Slice 4 (192.168.94.1) Code Option 2 yyy 0xC203 0x004,0xC401 0x004,0xC401 0xC203 0xC303 0x004,0xC402 0x004,0xC402 0xC303 0xC403 0x004,0xC403 0x004,0xC403 0xC403
Multi Slice IP Addresses for Test Packets VLAN UDP Rx DPort Tunnel SA Tunnel DA IP SA IP DA Proto SPort DPort 1 0xC100 192.168.81.1 192.168.91.1 192.168.81.2 0x11 2 0xC101 192.168.81.3 0xC102 192.168.81.4 0xC103 0xC200 192.168.82.1 192.168.92.1 192.168.82.2 0xC201 192.168.82.3 0xC202 192.168.82.4 0xC203 3 0xC300 192.168.83.1 192.168.93.1 192.168.83.2 0xC301 192.168.83.3 0xC302 192.168.83.4 0xC303 4 0xC400 192.168.84.1 192.168.94.1 192.168.84.2 0xC401 192.168.84.3 0xC402 192.168.84.4 0xC403
November Intel Demo 4 MRs 4 MIs per MR 4 QIDs per MI 4 Hosts Monitor: Numbered 1, 2, 3, 4 4 MIs per MR 4 QIDs per MI To the User: Numbered 1, 2, 3, 4 on each MI Internally to the NP filters: np_qid = 16 * (MR# - 1) + 4 * (MI# - 1 ) + USER_QID 4 Hosts coffee, momcat, tabby, foghorn Each with a presence on each of the MRs Monitor: Input BW and Pkts per MI at the LCI HF Pre-Q counters Output BW and Pkts per MI at the LCE HF Pre-Q counters Q-Length at the IPv4 MR QM LC Stats Index: Np_stats_index = 4 * (MR# - 1) + MI#
RTM Connections LC SWITCH? NPUB NPUA R T M R T M H4 H3 H2 H1 P10 P10
Demo Config MR-1 1 2 4 3 H1 H2 H4 H3 MR1_Flow12 MR1_Flow13 MR1_Flow14 H1 (coffee) 192.168.81.1 192.168.82.1 192.168.83.1 192.168.84.1 H2 (momcat) 192.168.81.2 192.168.82.2 192.168.83.2 192.168.84.2 H3 (tabby) 192.168.81.3 192.168.82.3 192.168.83.3 192.168.84.3 H4 (foghorn) 192.168.81.4 192.168.82.4 192.168.83.4 192.168.84.4 MR4_Flow21 MR4_Flow23 MR4_Flow24 1 2 MR-4 4 3 MR4_Flow41 MR4_Flow42 MR4_Flow43 MR4_Flow31 MR4_Flow32 MR4_Flow34 H4 H3
Tunnels, LCI Filters, QIDs and Stats Indices MR M1 VLAN UDP Rx DPort Tunnel SA Tunnel DA Stats Index QID 1 0xC100 192.168.81.1 192.168.91.1 2 0xC101 192.168.81.2 3 0xC102 192.168.81.3 4 0xC103 192.168.81.4 0xC200 192.168.82.1 192.168.92.1 5 0xC201 192.168.82.2 6 0xC202 192.168.82.3 7 0xC203 192.168.82.4 8 0xC300 192.168.83.1 192.168.93.1 9 0xC301 192.168.83.2 10 0xC302 192.168.83.3 11 0xC303 192.168.83.4 12 0xC400 192.168.84.1 192.168.94.1 13 0xC401 192.168.84.2 14 0xC402 192.168.84.3 15 0xC403 192.168.84.4 16
Tunnels, LCE Filters, QIDs and Stats Indices MR M1 VLAN UDP Rx DPort Tunnel SA Tunnel DA Stats Index QID 1 0xC100 192.168.91.1 192.168.81.1 2 0xC101 192.168.81.2 3 0xC102 192.168.81.3 4 0xC103 192.168.81.4 0xC200 192.168.92.1 192.168.82.1 5 0xC201 192.168.82.2 6 0xC202 192.168.82.3 7 0xC203 192.168.82.4 8 0xC300 192.168.93.1 192.168.83.1 9 0xC301 192.168.83.2 10 0xC302 192.168.83.3 11 0xC303 192.168.83.4 12 0xC400 192.168.94.1 192.168.84.1 13 0xC401 192.168.84.2 14 0xC402 192.168.84.3 15 0xC403 192.168.84.4 16
MR-1 Filters and QIDs DA SA MI USER_QID NP_QID 192.168.81.1 192.168.81.2 1 2 192.168.81.3 3 192.168.81.4 4 5 7 8 9 10 12 13 14 15
MR-2 Filters and QIDs DA SA MI USER_QID NP_QID 192.168.82.1 192.168.82.2 1 2 18 192.168.82.3 3 19 192.168.82.4 4 20 21 23 24 25 26 28 29 30 31
MR-3 Filters and QIDs DA SA MI USER_QID NP_QID 192.168.83.1 192.168.83.2 1 2 34 192.168.83.3 3 35 192.168.83.4 4 36 37 38 40 41 42 44 45 46 47
MR-4 Filters and QIDs DA SA MI USER_QID NP_QID 192.168.84.1 192.168.84.2 1 2 50 192.168.84.3 3 51 192.168.84.4 4 52 53 54 56 57 58 60 61 62 63
MR-1 Packet Flows VLAN UDP Rx DPort Tunnel SA Tunnel DA IP SA IP DA Proto SPort DPort 1 0xC100 192.168.81.1 192.168.91.1 192.168.81.2 0x11 2 192.168.81.3 3 192.168.81.4 4 0xC101 0xC102 0xC103
MR-2 Packet Flows VLAN UDP Rx DPort Tunnel SA Tunnel DA IP SA IP DA Proto SPort DPort 2 0xC200 192.168.82.1 192.168.92.1 192.168.82.2 0x11 1 192.168.82.3 3 192.168.82.4 4 0xC201 0xC202 0xC203
MR-3 Packet Flows VLAN UDP Rx DPort Tunnel SA Tunnel DA IP SA IP DA Proto SPort DPort 3 0xC300 192.168.83.1 192.168.93.1 192.168.83.2 0x11 2 1 192.168.83.3 192.168.83.4 4 0xC301 0xC302 0xC303
MR-4 Packet Flows VLAN UDP Rx DPort Tunnel SA Tunnel DA IP SA IP DA Proto SPort DPort 4 0xC400 192.168.84.1 192.168.94.1 192.168.84.2 0x11 2 1 192.168.84.3 3 192.168.84.4 0xC401 0xC402 0xC403
End of November Intel Demo Slides