Attilla de Groot | Sr. Systems Engineer, HCIE #3494 | Cumulus Networks EVPN to the host Host multitenancy Attilla de Groot Introduction Not a vendor EVPN pitch Attilla de Groot | Sr. Systems Engineer, HCIE #3494 | Cumulus Networks

Agenda EVPN to the Host Multi tenancy use cases Deployment issues Host integration with EVPN Caveats and future work Deployment scenarios from customer (field deployment experience) Issues with theses How to integrate it with EVPN Caveats and Future work

Multi tenancy use cases Virtualization environments VM environments MLAG to hypervisors Vxlan between hosts Double overlay Dedicated network nodes Typical VM environment with either a Vmware NSX or Openstack MLAG to a hypervisor with one or more vlans to this (can also be routing, but not frequently used) Double overlay if multiple vlans Dedicated network nodes that tenant traffic flows through

Multi tenancy use cases VM deployment issues Architecture issues MLAG to hypervisors Vxlan between hosts Double overlay Dedicated network nodes Orchestration problems MLAG to hypervisors, not a standard, still layer2, host configuration? VxLAN between hosts ignores underlay, traffic tromboning etc. If multiple VLANs, double overlay with Vxlan in ”underlay” Dedicated network nodes create their own issues: redundancy, latency, bottleneck Orchestration, who manages what? Network integration and a bare metal host, OVSDB … etc.

Multi tenancy use cases Container environments BGP to hosts Host route advertisement Docker management Container overlay Container deployments already do routing to hosts Advertise the hosts (or container) IPs into the fabric Sometimes a container overlay with VxLAN as well (using dedicated network node again)

Multi tenancy use cases Container deployment issues Architecture issues Host networking Multi-tenancy IP prefix overlap ACL management Who manages the host networking part? Multitenancy on hosts, dedicate host to customer? No IP prefix multi-tenancy ACLs for tenant separation

Host integration with EVPN Open standards EVPN on hosts Vlan aware bridge VRF in Linux kernel VxLAN Free Range Routing with EVPN Ifupdown2 Iproute2 cumulus@server01:~$ uname -a Linux server01 4.17.0-041700-generic #201806041953 SMP Mon Jun 4 19:55:25 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux cumulus@server01:~$ vrf list VRF              Table ---------------- ----- mgmt              1001 tenant3           1002 tenant2           1003 tenant1           1004 Quite some changes and updates on the Linux networking over the past few years Vlan aware bridge, mostly an valid for a whitebox environment, but also easier to configure and maintain VRFs natively in the Linux kernel (>= 4.10) Kernel VxLAN support Free range routing has support for EVPN, type 2,3 and 5 Tools like ifupdown2 and iproute2

Host integration with EVPN Host connectivity auto eth1 iface eth1 auto eth2 iface eth2 router bgp 65501  bgp router-id 10.250.250.1  bgp bestpath as-path multipath-relax  neighbor FABRIC peer-group  neighbor FABRIC remote-as external  neighbor FABRIC timers 1 3  neighbor eth1 interface peer-group FABRIC  neighbor eth2 interface peer-group FABRIC  !  address-family ipv4 unicast   redistribute connected route-map loopbacks  exit-address-family  address-family l2vpn evpn    neighbor FABRIC activate    advertise-all-vni    advertise ipv4 unicast    advertise ipv6 unicast ! route-map loopbacks permit 10  match interface lo Host connectivity Routing to hosts BGP unnumbered RFC5549 Loopback advertisement EVPN address family interface eth1   ipv6 nd ra-interval 10  no ipv6 nd suppress-ra interface eth2  What does it get you? Always routing to hosts with BGP, no more MLAG and Layer2 FRR has support for BGP unnumbered, easy configuration Assuming that network does do EVPN, advertise loopback and enable address family

Host integration with EVPN L2 tenancy Vlan-aware bridge ARP / ND suppression L2VNIs EVPN Type-2 L2 requirement for VMs Build L2VNIs, advertised over EVPN Advertise MAC/IP routes Control over broadcast domain with ARP / ND suppression

Host integration with EVPN Distributed routing Gateway location SVIs on host Anycast gateway Local host routing Where to start routing? Centralized? Solving the networking node issue. EVPN & Distributed routing with anycast gateway Start routing on the host by configuring an SVI in the local vlan

Host integration with EVPN L3 tenancy VRFs on host Prefix advertisement L3VNIs EVPN Type-5 Multiple tenants on hosts with L3 separation Use Linux VRFs in combination with L3VNIs Announce prefixes with EVPN type5

Host integration with EVPN Container integration Container IP redistribution Host route advertisement Prefix overlap No ACLs for tenant segregation Container multitenancy, how to do? Redistribute container IPs in specific VRF (typically through the docker API) Allow for prefix overlap and remove the need for ACLs for segregation

Host integration with EVPN Bare metal integration Integration with RFC standard L2 stretching L3 tenancy Distributed routing Integrate bare metal hosts with single protocol and anycast VTEP Allow for distributed routing for both solutions

Future work Proof of technology cumulus@server01:~$ uname -a Linux server01 4.17.0-041700-generic #201806041953 SMP Mon Jun 4 19:55:25 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Proof of technology Commercial support Software / tools availability Linux kernel FRR Demo cumulus@server01:~$ bridge vlan port vlan ids bridge 1 PVID Egress Untagged 1010 1011 1012 1013 1014 cumulus@server01:~$ vrf list VRF              Table ---------------- ----- mgmt              1001 tenant3           1002 tenant2           1003 tenant1           1004 cumulus@server01:~$ sudo vtysh -c "show bgp evpn vni"   VNI        Type RD                    Import RT                 Export RT                 Tenant VRF                             * 101010     L2   10.250.250.1:5        65501:101010              65501:101010             tenant1                               * 101012     L2   10.250.250.1:3        65501:101012              65501:101012             tenant2                               * 101014     L2   10.250.250.1:7        65501:101014              65501:101014             tenant3 * 404001     L3   172.30.11.1:8         65501:404001              65501:404001             tenant1                               * 404002     L3   172.30.13.1:9         65501:404002              65501:404002             tenant2                               * 404003     L3   172.30.15.1:10        65501:404003              65501:404003             tenant3  While available right now (demo), this is all quite new No commercial support for FRR, new kernel and FRR Sevaral tools combined https://gitlab.com/CumulusNetworks/evpn-to-the-host

Future work Orchestration & Caveats Openstack neutron Kubernetes / Swarm Host / Network management Head-end replication BUM anycast Merchant silicon limitations Multicast replication PIM-SM Integration with orchestration tools Openstack neutron driver to control FRR, bridge VNI configuration? Who (tools) manages the hosts and network parts? BUM anycast replication with Merchant silicon Multicast replication according to VxLAN RFC Planned to be implemented in FRR … PIM-SM on hosts?

Future work Route leaking & Micro segmentation Inter tenant traffic Service tenant FRR implementation Micro segmentation Host traffic filtering Filtering with BPF Flowspec for ACL distribution If there is the requirement for inter tenant traffic … through router on a stick or firewall “Service” tenant for services like DNS or general tenant services Not yet implemented in FRR Micro segmentation inside a tenant for application security Kernel development for doing filtering with BPF on hosts Using flowspec for distributing the ACLs

EVPN to the Host Questions ?