1 © OneCloud and/or its affiliates. All rights reserved. VXLAN Overview Module 4

2 © OneCloud and/or its affiliates. All rights reserved.  VXLAN Benefits  VXLAN Implementation  VXLAN-Peer and Host discovery  VXLAN Packet Forwarding  VXLAN Today

3 © OneCloud and/or its affiliates. All rights reserved. VXLAN Ethernet in IP overlay network  Entire Layer 2 frame encapsulated in User Datagram Protocol (UDP)  50 bytes of overhead Include 24-bit VXLAN identifier  16 M logical networks  Mapped into local bridge domains Tunnel Between VEMs Two Options for BUM traffic  IP Multicast  Ingress replication IP multicast used for Layer 2 broadcast or multicast, and unknown unicast Leverages Layer-3 ECMP – all links forwarding Integration of Physical and Virtual

4 © OneCloud and/or its affiliates. All rights reserved. VXLAN Underlay Network IP routed Network  Support any routing protocols --- OSFP, EIGRP, IS-IS, BGP, etc. IP Transport Network  Flexible topologies  Recommend a network with redundant paths using ECMP for load sharing  Multicast is needed if using multicast for overlay BUM replication and transport

5 © OneCloud and/or its affiliates. All rights reserved. VXLAN VTEP  VXLAN terminates its tunnels on VTEPs (Virtual Tunnel End Point).  Each VTEP has two interfaces, one is to provide bridging function for local hosts, the other has an IP identification in the core network for VXLAN encapsulation/ decapsulation. Local LAN Segment IP Interface End System VTEP Transport IP Network Local LAN Segment IP Interface End System VTEP

6 © OneCloud and/or its affiliates. All rights reserved. VXLAN Frame Format MAC-in-IP Encapsulation Underlay Outer IP Header Outer MAC Header UDP Header VXLAN Header Original Layer-2 Frame Overlay 14 Bytes (4 Bytes Optional) Ether Type 0x0800 Ether Type 0x0800 VLAN ID Tag VLAN ID Tag VLAN Type 0x8100 VLAN Type 0x8100 Src. MAC Address Dest. MAC Address Bytes Dest. IP Source IP Header Checksum Header Checksum Protocol 0x11 (UDP) IP Header Misc. Data IP Header Misc. Data Bytes Checksum 0x0000 UDP Length VXLAN Port Source Port Source Port 16 8 Bytes Reserved VNI Reserved VXLAN Flags RRRRIRRR Src VTEP MAC Address Next-Hop MAC AddressSrc and Dst addresses of the VTEPs Allows for 16M possible Segments UDP 4789 Hash of the inner L2/L3/L4 headers of the original frame. Enables entropy for ECMP Load balancing in the Network. 50 (54) Bytes of Overhead

7 © OneCloud and/or its affiliates. All rights reserved. VXLAN Tunnel End Point (VTEP) IP B IP c Virtual Switch VXLAN ORANGE IP A Virtual Switch VTEP  VTEP is the IP address which defines the Tunnel End Point  VTEP represents source and destination IP address on a VXLAN encapsulated tunnel  VTEP would reside on a switch (physical or virtual) which would perform the VXLAN encapsulation VXLAN ORANGE VTEP

8 © OneCloud and/or its affiliates. All rights reserved. VXLAN Evolution  Yesterday: VXLAN, yet another Overlay  Data-Plane only (Multicast based Flood & Learn)  Today: VXLAN for the creation of scalable DC Fabrics – Intra-DC  Control-Plane, active VTEP discovery, Multicast and Unicast (Head-End Replication)  Future: VXLAN for DCI – Inter-DC  DCI Enhancements (ARP caching/suppress, Multi-Homing, Failure Domain isolation, Loop Protection etc.)

9 © OneCloud and/or its affiliates. All rights reserved.

10 © OneCloud and/or its affiliates. All rights reserved. VXLAN Overview (1) Local LAN Segment Physical Host Local LAN Segment Physical Host Virtual Hosts Local LAN Segment Virtual Switch Edge Device IP Interface

11 © OneCloud and/or its affiliates. All rights reserved. VXLAN Overview (2) Local LAN Segment Physical Host Local LAN Segment Physical Host VTEP Encapsulation Virtual Hosts Local LAN Segment Virtual Switch VTEP – VXLAN Tunnel End-Point VNI/VNID – VXLAN Network Identifier V V V

12 © OneCloud and/or its affiliates. All rights reserved.

13 © OneCloud and/or its affiliates. All rights reserved. VXLAN Peer and Host Learning Options Host LearningData-PlaneControl-Plane Core Multicast Unicast Vlan 2 vn-segment 4098 Interface nve 1 host-reachability protocol bgp member vni 4098 ingress-replication protocol bgp Vlan 2 vn-segment 4098 Interface nve 1 host-reachability protocol bgp member vni 4098 ingress-replication protocol bgp Vlan 2 vn-segment 4098 Interface nve 1 member vni 4098 ingress-replication protocol static Vlan 2 vn-segment 4098 Interface nve 1 member vni 4098 ingress-replication protocol static Vlan 2 vn-segment Interface nve 1 host-reachability protocol bgp member vni 4098 mcast-group Vlan 2 vn-segment Interface nve 1 host-reachability protocol bgp member vni 4098 mcast-group Vlan 2 vn-segment 4098 Interface nve 1 member vni mcast-group Vlan 2 vn-segment 4098 Interface nve 1 member vni mcast-group Flood and Learn Peer Learning: DP EVPN-Multicast Peer Learning: BGP Static Ingress-Replication Peer Learning: CLI EVPN Ingress-Replication Peer Learning: BGP

14 © OneCloud and/or its affiliates. All rights reserved. VXLAN Flood & Learn V1V1 V3V3 Host B MAC_B / IP_B MACVNIVTEP MAC_B30000E1/4 MACVNIVTEP MAC_C30000E1/8 V2V2 Host A MAC_A / IP_A Virtual Switch Host C MAC_C / IP_C MACVNIVTEP MAC_A30000E1/12

15 © OneCloud and/or its affiliates. All rights reserved. VTEP Peer Discovery & Address Learning (1) VXLAN Flood & Learn V1V1 V3V3 Underlay SIP: IP_V1 DIP: SIP: IP_V1 DIP: SMAC: MAC_V1 DMAC: 00:01:5E:01:01:01 SMAC: MAC_V1 DMAC: 00:01:5E:01:01:01 UDP VXLAN VNID: ARP Request SMAC: MAC_A DMAC: FF:FF:FF:FF:FF:FF SMAC: MAC_A DMAC: FF:FF:FF:FF:FF:FF Overlay 2 MACVNIVTEP MAC_A30000E1/12 MACVNIVTEP MAC_B30000E1/4 MAC_A30000IP_V1 Virtual Switch ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 4 MACVNIVTEP MAC_C30000E1/8 MAC_A30000IP_V1 V2V2 3 1 ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 3 ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 4 MACVNIVTEP MAC_B30000E1/4 MACVNIVTEP MAC_C30000E1/8 Host A MAC_A / IP_A Host C MAC_C / IP_C Host B MAC_B / IP_B

16 © OneCloud and/or its affiliates. All rights reserved. VTEP Peer Discovery & Address Learning (2) VXLAN Flood & Learn V3V3 ARP Response from IP_B Src MAC: MAC_B Dst MAC: MAC_A 5 ARP Response for IP_B Src MAC: MAC_B Dst MAC: MAC_A 7 V2V2 V1V1 Underlay SIP: IP_V2 DIP: IP_V1 SIP: IP_V2 DIP: IP_V1 SMAC: MAC_V2 DMAC: hop-by-hop SMAC: MAC_V2 DMAC: hop-by-hop UDP VXLAN VNID: ARP Response SMAC: MAC_B DMAC: MAC_A SMAC: MAC_B DMAC: MAC_A Overlay 6 Host A MAC_A / IP_A Host B MAC_B / IP_B MACVNIVTEP MAC_B30000E1/4 MAC_A30000IP_V1 MACVNIVTEP MAC_A30000E1/12 MACVNIVTEP MAC_A30000E1/12 MAC_B30000IP_V2

17 © OneCloud and/or its affiliates. All rights reserved. VTEP Peer Discovery & Address Learning (3) VXLAN Flood & Learn Host X MAC_X / IP_X Virtual Switch V1V1 V3V3 V2V2 ARP Request for IP_Y Src MAC: MAC_X Dst MAC: FF:FF:FF:FF:FF:FF 4 ARP Request for IP_Y Src MAC: MAC_X Dst MAC: FF:FF:FF:FF:FF:FF 1 Underlay SIP: IP_V1 DIP: SIP: IP_V1 DIP: SMAC: MAC_V1 DMAC: 00:01:5E:01:01:02 SMAC: MAC_V1 DMAC: 00:01:5E:01:01:02 UDP VXLAN VNID: ARP Request SMAC: MAC_X DMAC: FF:FF:FF:FF:FF:FF SMAC: MAC_X DMAC: FF:FF:FF:FF:FF:FF Overlay 2 3 MACVNIVTEP MAC_Y30001E1/8 MAC_X30001V1 MACVNIVTEP MAC_X30001E1/11 Host Y MAC_Y / IP_Y

18 © OneCloud and/or its affiliates. All rights reserved. VTEP Peer Discovery & Address Learning (4) Host X MAC_X / IP_X VXLAN Flood & Learn MACVNIVTEP MAC_X30001E1/11 MAC_Y30001V3 V2V2 V1V1 Virtual Switch ARP Response for IP_Y Src MAC: MAC_Y Dst MAC: MAC_X 5 V3V3 MACVNIVTEP MAC_Y30001E1/8 MAC_X30001V1 7 ARP Response for IP_Y Src MAC: MAC_Y Dst MAC: MAC_X Underlay SIP: IP_V3 DIP: IP_V1 SIP: IP_V3 DIP: IP_V1 SMAC: MAC_V3 DMAC: hop-by-hop SMAC: MAC_V3 DMAC: hop-by-hop UDP VXLAN VNID: ARP Response SMAC: MAC_Y DMAC: MAC_X SMAC: MAC_Y DMAC: MAC_X Overlay 6 Host Y MAC_Y / IP_Y MACVNIVTEP MAC_X30001E1/11

19 © OneCloud and/or its affiliates. All rights reserved.

20 © OneCloud and/or its affiliates. All rights reserved. VXLAN Packet Forwarding (1) VXLAN Flood & Learn Host A MAC_A / IP_A Host B MAC_B / IP_B V3V3 4 MACVNIVTEP MAC_B30000E1/4 MAC_A30000V1V1 MACVNIVTEP MAC_A30000E1/12 MAC_B30000V2 V2V2 V1V1 SIP: IP_A DIP: IP_B SIP: IP_A DIP: IP_B SMAC: MAC_A DMAC: MAC_B SMAC: MAC_A DMAC: MAC_B 1 SIP: IP_A DIP: IP_B SIP: IP_A DIP: IP_B SMAC: MAC_A DMAC: MAC_B SMAC: MAC_A DMAC: MAC_B Underlay SIP: IP_V1 DIP: IP_V2 SIP: IP_V1 DIP: IP_V2 SMAC: MAC_V1 DMAC: hop-by-hop SMAC: MAC_V1 DMAC: hop-by-hop UDP VXLAN VNID: SMAC: MAC_A DMAC: MAC_B SMAC: MAC_A DMAC: MAC_B SIP: IP_A DIP: IP_B SIP: IP_A DIP: IP_B Overlay 2 SIP: IP_V1 DIP: IP_V2 SIP: IP_V1 DIP: IP_V2 SMAC: hop-by-hop DMAC: MAC_V2 SMAC: hop-by-hop DMAC: MAC_V2 Underlay VXLAN VNID: SMAC: MAC_A DMAC: MAC_B SMAC: MAC_A DMAC: MAC_B SIP: IP_A DIP: IP_B SIP: IP_A DIP: IP_B UDP Overlay 3

21 © OneCloud and/or its affiliates. All rights reserved. VXLAN Packet Forwarding (2) VXLAN Flood & Learn V2V2 Underlay SIP: IP_V1 DIP: IP_V3 SIP: IP_V1 DIP: IP_V3 SMAC: MAC_V1 DMAC: hop-by-hop SMAC: MAC_V1 DMAC: hop-by-hop UDP VXLAN VNID: SMAC: MAC_X DMAC: MAC_Y SMAC: MAC_X DMAC: MAC_Y SIP: IP_X DIP: IP_Y SIP: IP_X DIP: IP_Y Overlay 2 Virtual Switch MACVNIVTEP MAC_Y30001E1/8 MAC_X30001V1 Host X MAC_X / IP_X 1 MACVNIVTEP MAC_X30001E1/11 MAC_Y30001V3 SIP: IP_X DIP: IP_Y SIP: IP_X DIP: IP_Y SMAC: MAC_X DMAC: MAC_Y SMAC: MAC_X DMAC: MAC_Y V3V3 V1V1 4 SIP: IP_X DIP: IP_Y SIP: IP_X DIP: IP_Y SMAC: MAC_X DMAC: MAC_Y SMAC: MAC_X DMAC: MAC_Y Underlay SIP: IP_V1 DIP: IP_V3 SIP: IP_V1 DIP: IP_V3 SMAC: MAC_V1 DMAC: MAC_V3 SMAC: MAC_V1 DMAC: MAC_V3 UDP VXLAN VNID: SMAC: MAC_X DMAC: MAC_Y SMAC: MAC_X DMAC: MAC_Y SIP: IP_X DIP: IP_Y SIP: IP_X DIP: IP_Y Overlay 3 Host Y MAC_Y / IP_Y

22 © OneCloud and/or its affiliates. All rights reserved. VXLAN Evolution  Leveraging the Control-Plane to avoid Flood & Learn VTEP discovery (pro-active learning)  Head-End Replication to relax the requirement for Multicast in the Underlay 22 Multicast Independent* Overlay Control-Plane provides dynamic VTEP discovery Head-End Replication enables Unicast-only mode (aka ingress Replication) Overlay Control-Plane provides dynamic VTEP discovery Head-End Replication enables Unicast-only mode (aka ingress Replication) *Multicast Independence requires the usage of the Overlay Control-Plane or static configuration

23 © OneCloud and/or its affiliates. All rights reserved. Head-End Replication Multicast Independent 23 Host A MAC_A / IP_A Host B MAC_B / IP_B Virtual Switch ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 5 ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 5 Underlay SIP: IP_V1 DIP: IP_V3 SIP: IP_V1 DIP: IP_V3 SMAC: MAC_V1 DMAC: hop-by-hop SMAC: MAC_V1 DMAC: hop-by-hop UDP VXLAN VNID: ARP Request SMAC: MAC_A DMAC: FF:FF:FF:FF:FF:FF SMAC: MAC_A DMAC: FF:FF:FF:FF:FF:FF Overlay 4 Host C MAC_C / IP_C PeerVNIVTEP V V1 V230000V2 RR V2V2 PeerVNIVTEP V130000V1V1 V V3 V1V1 V3V3 PeerVNIVTEP V230000V2 V V3 1 ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 2 Underlay SIP: IP_V1 DIP: IP_V2 SIP: IP_V1 DIP: IP_V2 SMAC: MAC_V1 DMAC: hop-by-hop SMAC: MAC_V1 DMAC: hop-by-hop UDP VXLAN VNID: ARP Request SMAC: MAC_A DMAC: FF:FF:FF:FF:FF:FF SMAC: MAC_A DMAC: FF:FF:FF:FF:FF:FF Overlay 4 3

24 © OneCloud and/or its affiliates. All rights reserved. VXLAN Evolution  Multi-Protocol BGP (MP-BGP) based Control-Plane using EVPN NLRI (Network Layer Reachability Information)  Make Forwarding decisions at VTEPs for Layer-2 (MAC) and Layer-3 (IP); Integrated Route/Bridge (IRB)  Reduce Flooding  Reduce impact of ARP on the Network  Standards Based (IETF draft) Protocol Learning Workload MAC and IP Addresses learnt by VXLAN Edge Devices (NVEs) Advertises Layer-2 and Layer-3 Address-to-VTEP Association (Overlay Control-Plane) Flood Prevention Optimized ARP forwarding Workload MAC and IP Addresses learnt by VXLAN Edge Devices (NVEs) Advertises Layer-2 and Layer-3 Address-to-VTEP Association (Overlay Control-Plane) Flood Prevention Optimized ARP forwarding

25 © OneCloud and/or its affiliates. All rights reserved. Terminology - Reference Layer-2 VNI  VNI (VXLAN network identifier) carried in VXLAN packets bridged across VTEPs (VXLAN tunnel end point). This VNI is configured per VLAN. Layer-3 VNI  VNI carried in the VxLAN packets routed across VTEPs. This VNI is linked per Tenant VRF. Anycast GW  All L3 VTEPs are configured with same mac and same subnet for host facing SVI. VRF overlay VLAN  Every Tenant VRF will need a Vlan to be configured for VXLAN routing.  This VLAN is configured with L3-VNI. VXLAN L2 Gateway  VTEP capable of switching VLAN->VXLAN, VXLAN->VLAN packets with in same VNI. VXLAN L3 Gateway  VTEP capable of routing packets across different VNIs.

26 © OneCloud and/or its affiliates. All rights reserved. Thank You