Design of a Diversified Router: Packet Formats John DeHart jdd@arl.wustl.edu http://www.arl.wustl.edu/arl

Revision History 6/2/06 (JDD): 6/6/06 (JDD): 6/13/06 (JDD): Created Modified Internal Frame formats Added some notes about handling of frames with IPv4 MR Internal Headers. 6/13/06 (JDD): Updated notes on handling frames with IPv4 MR Internal Headers.

External Frame Formats: with VLAN DstAddr (6B) Type=IP (2B) PAD (nB) CRC (4B) IP Payload Dst Addr (4B) Src Addr (4B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol (1B) Hdr Cksum (2B) Type=802.1Q (2B) TCI ≠ VLAN0 (2B) DstAddr (6B) SrcAddr (6B) Legacy SrcAddr (6B) Type=802.1Q (2B) TCI ≠ VLAN0 (2B) Type=802.1Q (2B) MLI (2B) LEN (2B) Meta Frame TCI (2B) Type=Substrate (2B) PAD (nB) CRC (4B) DstAddr (6B) SrcAddr (6B) Type=IP (2B) Type=802.1Q (2B) MLI (2B) LEN (2B) Meta Frame TCI=VLAN0 (2B) Type=Substrate (2B) PAD (nB) CRC (4B) DstAddr (6B) SrcAddr (6B) P2P-VLAN0 Multi-Access Type=802.1Q (2B) MLI (2B) LEN (2B) Meta Frame TCI≠VLAN0 (2B) Type=Substrate (2B) PAD (nB) CRC (4B) DstAddr (6B) SrcAddr (6B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) TTL (1B) Protocol=Substrate (1B) Hdr Cksum (2B) Src Addr (4B) Dst Addr (4B) MLI (2B) LEN (2B) Meta Frame PAD (nB) CRC (4B) P2P-DC (Configured) P2P-Tunnel Blue Shading: Determine SL Type Black Outline: Key Fields from pkt

External Frame Formats: w/o VLAN DstAddr (6B) DstAddr (6B) SrcAddr (6B) SrcAddr (6B) Type=IP (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) Ver/HLen/Tos/Len (4B) DstAddr (6B) DstAddr (6B) ID/Flags/FragOff (4B) ID/Flags/FragOff (4B) TTL (1B) SrcAddr (6B) TTL (1B) Protocol=Substrate (1B) SrcAddr (6B) Protocol (1B) Hdr Cksum (2B) Type=Substrate (2B) Hdr Cksum (2B) Src Addr (4B) Type=Substrate (2B) MLI (2B) Src Addr (4B) MLI (2B) LEN (2B) Dst Addr (4B) Dst Addr (4B) LEN (2B) Meta Frame MLI (2B) IP Payload Meta Frame LEN (2B) Meta Frame PAD (nB) PAD (nB) PAD (nB) PAD (nB) CRC (4B) CRC (4B) CRC (4B) CRC (4B) P2P-DC (Configured) P2P-Tunnel Legacy Multi-Access Blue Shading: Determine SL Type Black Outline: Key Fields from pkt

External Frame Formats: GRE We are considering GRE tunnels as another SL Type DstAddr (6B) SrcAddr (6B) DstAddr (6B) Type=802.1Q (2B) SrcAddr (6B) TCI ≠ VLAN0 (2B) Type=IP (2B) Type=IP (2B) Ver/HLen/Tos/Len (4B) Ver/HLen/Tos/Len (4B) ID/Flags/FragOff (4B) ID/Flags/FragOff (4B) TTL (1B) TTL (1B) Protocol=GRE (1B) Protocol=GRE (1B) Hdr Cksum (2B) Hdr Cksum (2B) Src Addr (4B) ` Src Addr (4B) Dst Addr (4B) Dst Addr (4B) Flags/recur/Ver (2B) Flags/recur/Ver (2B) ` Type=Substrate (2B) Type=Substrate (2B) ` Optional Fields (nB) Optional Fields (nB) MLI (2B) ` MLI (2B) LEN (2B) LEN (2B) Meta Frame Meta Frame MLI (2B) PAD (nB) PAD (nB) CRC (4B) CRC (4B) Blue Shading: Determine SL Type Black Outline: Key Fields from pkt VLAN GRE GRE

Internal Frame Format DAddr (6B) SAddr (6B) Ethernet Header 18 Bytes Type=802.1Q (2B) VLAN (2B) Type=Substrate (2B) Ethernet Header 18 Bytes Substrate Header Metanet Frame (Metanet Frame Len Bytes) Type (4b) Metanet Frame Len (12b) Ethernet Trailer (Pad+CRC) Destination MPE/MI (2B) 6-70 Bytes Source MPE/MI (2B) Type Dependent Data (0-64B) Type:Defines what type of frame we have See next slides for details on types MPE: Meta Processing Element MI: Meta Interface MetaNet Frame Length: Length of Metanet Frame in Bytes

Substrate Header Ingress: LC  MPE Transit: LC  LC Destination MPE (2B) RxMI (2B) Metanet Frame Len (12b) 0000 Ingress: LC  MPE Transit: LC  LC TxMI and RxMI are defined by the Substrate on the LC They are not actual MIs on a MR TxMI (2B) RxMI (2B) Metanet Frame Len (12b) 0001 Destination MPE (2B) Source MPE (2B) Metanet Frame Len (12b) 0010 Internal: MPE  MPE MNs can define their on MN Internal header to be put in the MetaNet Frame TxMI (2B) Source MPE (2B) Metanet Frame Len (12b) 0100 Egress Simple: MPE  LC No extra information needed

Substrate Header (continued) TxMI (2B) Source MPE (2B) Metanet Frame Len (12b) 0101 Length (1B) NH MN Addr (0-63B) Egress NH_MN_Addr: MPE  LC MPE provides a next hop MN address for Substrate to perform ARP for it. (Length + Nh MN Addr) field padded to be even number of bytes long TxMI (2B) Source MPE (2B) Metanet Frame Len (12b) 0110 MAC Address (6B) Egress MAC_ADDR: MPE  LC MPE provides MAC address for Substrate to use when sending out on Egress link

Type Dependent Data (0-???B) IPv4 MR Internal Header Length (1B) Exception Type (1B) MPE to CP Slow path handling of exception packets Some exceptions may require additional data to be communicated to CP RxMI (2B) Type Dependent Data (0-???B) CP to MPE !Reclassify Return packets to MPE for re-injection into fast path Also used by CP to inject new packets into data stream. For example, ICMP MR does not need to reclassify Substrate would need a lookup on TxMI to get the information needed to get packet to LC. Length = 4 Bytes Type = 0x0 No Type Dependent Data TxMI (2B) Length (1B) Type (1B) Type Dependent Data (0-???B) RxMI (2B) Length (1B) Type (1B) Type Dependent Data (0-???B) CP to MPE Reclassify Return packets to MPE for re-injection into fast path Ingress Lookup: Redo the normal lookup with provide RxMI in Key. Length = 4 Bytes Type = 0x1 No Type Dependent Data

IPv4 MR Internal Header Notes Input data to Parse is MN Frame Offset and Length Output data from Parse is IP Pkt Offset and Length If there is a MN Internal Header, Parse basically consumes it. Hdr Format needs to start at the beginning of the IP Header and re-write headers upward in the Buffer. If Parse receives a frame/pkt from the CP, It should not decrement the TTL. Assume a pkt coming from the CP has the correct TTL for going out on the link It should validate the header and check the header checksum It should not generate an exception for IP options. Assume the CP has dealt with all the exceptions. What about the case where the TxMI is on a Multi-Access Link and needs a Next Hop MetaNet Address to resolve the next hop destination? For Frame/Pkt from CP, what does Demux put in RxMI data field passed to Parse?

LC  MPE (Ingress) Internal Format DAddr (6B) SAddr (6B) Type=802.1Q (2B) VLAN (2B) Type=Substrate (2B) Destination MPE (2B) RxMI (2B) Metanet Frame Len (12b) 0000 Metanet Frame (Metanet Frame Len Bytes) MPE Ethernet Trailer (Pad+CRC) Destination MPE: ID of receiving MPE RxMI: Receive Meta Interface

LC  LC (Transit) Internal Format DAddr (6B) LC SAddr (6B) Type=802.1Q (2B) VLAN (2B) Type=Substrate (2B) TxMI (2B) RxMI (2B) Metanet Frame Len (12b) 0001 Metanet Frame (Metanet Frame Len Bytes) LC Ethernet Trailer (Pad+CRC) TxMI: Substrate allocated MI for translation to Egress MLI RxMI: Substrate allocated MI Probably not really needed, present for symmetry

MPE  MPE Internal Format DAddr (6B) MPE SAddr (6B) Type=802.1Q (2B) VLAN (2B) Type=Substrate (2B) Destination MPE (2B) Source MPE (2B) Metanet Frame Len (12b) 0010 Metanet Frame (Metanet Frame Len Bytes) MPE Ethernet Trailer (Pad+CRC) Destination MPE: ID of receiving MPE Source MPE: ID of sending MPE MR specific information, things like exception packet indicators, etc. go in a MetaNet specific header in the MetaNet Frame.

MPE  LC (Egress Simple) Internal Format DAddr (6B) MPE SAddr (6B) Type=802.1Q (2B) VLAN (2B) Type=Substrate (2B) TxMI (2B) Source MPE (2B) Metanet Frame Len (12b) 0100 Metanet Frame (Metanet Frame Len Bytes) LC Ethernet Trailer (Pad+CRC) Source MPE: ID of sending MPE TxMI: Transmit Meta Interface

MPE  LC (Egress NH MN) Internal Format DAddr (6B) MPE SAddr (6B) Type=802.1Q (2B) VLAN (2B) Type=Substrate (2B) TxMI (2B) Source MPE (2B) Metanet Frame Len (12b) 0101 Length (1B) NH MN Addr (0-63B) LC Metanet Frame (Metanet Frame Len Bytes) Ethernet Trailer (Pad+CRC) Source MPE: ID of sending MPE TxMI: Transmit Meta Interface NH MN Addr: Padded to so that Length + NH MN Addr is an even number of Bytes

MPE  LC (Egress MAC) Internal Format DAddr (6B) MPE SAddr (6B) Type=802.1Q (2B) VLAN (2B) Type=Substrate (2B) TxMI (2B) Source MPE (2B) Metanet Frame Len (12b) 0110 MAC Address (6B) LC Metanet Frame (Metanet Frame Len Bytes) Ethernet Trailer (Pad+CRC) Source MPE: ID of sending MPE TxMI: Transmit Meta Interface MAC: We are assuming Ethernet addresses here. Should we?

Data Delivered by Substrate to MPE Metanet Frame Len (16b) Destination MPE (2B) RxMI (2B) Metanet Frame (Metanet Frame Len Bytes) In a shared NPU case: MR will not receive the Ethernet header or trailer Type field will be zeroed out and look like it is part of the Frame Length. Destination MPE should be id of MR MPE receiving packet

Extra The next set of slides are for templates or extra information if needed

Text Slide Template

Image Slide Template

IPv4 MR Internal Header Notes Input data to Parse is MN Frame Offset and Length Output data from Parse is IP Pkt Offset and Length If there is a MN Internal Header, Parse basically consumes it. Hdr Format needs to start at the beginning of the IP Header and re-write headers upward in the Buffer. If Parse receives a !Reclassify frame/pkt from the CP, it should not decrement the TTL. Assume this was done on the first trip through the MR. it should not generate exceptions again. For Frame/Pkt from CP, what does Demux put in RxMI data field passed to Parse?