7th OpenSTACK USER group nordics OpenStack Neutron Spyridon Dosis Spyridon.dosis@gmail.com 7th OpenSTACK USER group nordics

Outline Neutron Architecture Neutron and Linux Networking Neutron and OpenvSwitch Network Packet Flows Distributed Virtual Router

Neutron Overview OpenStack’s networking component, networking-as-a-service Create, configure, manage software-defined networks First introduced in Havana release, renamed from ‘Quantum’ Pluggable architecture enabling additional services e.g. load-balancing-as-a-service (LBaaS), VPN-as-a-service (VPNaaS), DNS-as-a-service (DNSaaS) interface with L2 & L3 hardware, SDN controllers etc.

Neutron in OpenStack Neutron uses Keystone (Identity Service) for authenticating / authorizing all API requests Integration with Horizon enabling tenants to create networks / subnets and cloud users to connect instances to tenant / provider networks through virtual NICs Interaction with Nova for the required ‘plugging’ of the virtual NICs to the VM instances

Neutron Terminology Provider Network Tenant Network Networks defined in OpenStack that map to the existing data center’s physical network Support for either flat or VLAN network types Can be shared among the tenants Tenant Network Networks defined by OpenStack tenants’ users Support for VLAN and GRE Route between tenant networks using the L3 agent and the Neutron routers Image taken from https://developer.rackspace.com/blog/neutron-networking-the-building-blocks-of-an-openstack-cloud/

A Basic Setup Demo Setup in VMWare Workstation Management Network (10.0.0.0/24) 10.0.0.51 controller 10.0.0.61 network 10.0.0.71 computeA 10.0.0.91 compute Data Network (10.0.1.0/24) 10.0.1.161 network 10.0.1.171 computeA 10.0.1.191 compute External Network (192.168.254.0/24) 192.168.254.129 controller 192.168.254.131 computeA 192.168.254.133 computeC Image taken from http://docs.openstack.org/havana/install-guide/install/yum/content/figures/3/a/common/figures/Neutron-PhysNet-Diagram.png

Tenant Network Features Support for Overlapping IPs Multiple Tenants having the same internal subnets (e.g. 192.168.1.0/24) but still remain isolated Configuration DHCP / Host routes / DNS name servers Metadata (e.g. SSH-key injection) Security Groups Ingress / Egress Rule Configuration Floating IPs (1:1 NAT) Support for IPv6

Neutron Component Communication Database neutron-server Message Queue L2 Agent(s) L3 / DHCP / Other Agents

Neutron Components neutron-server Implements the REST API (v2) calls Interfaces with other OpenStack services via RPC calls Database persistence Translates between API and plugin calls The plugin implements the core networking (network ports, networks and subnets, IP addressing etc.) Either implements directly or proxies to dedicated 3rd party hardware/software solution

ML2 Plugin A modular framework replacing previous monolithic plugins (openvswitch, linuxbridge) Network Type Drivers Flat, GRE , VLAN, VXLAN TypeDrivers Network Mechanism Drivers OpenvSwitch , Linuxbridge, Cisco Nexus etc. Allows multiple mechanisms to be used simultaneously

Open vSwitch An open source, software-based virtual switch Supports distribution across multiple physical servers Supports many features e.g. VLAN,GRE, VXLAN, NetFlow, sFlow, LACP The OpenvSwitch plugin is loaded by the Neutron service Receives API calls and stores the corresponding network data in a database backend The OpenvSwitch agent that runs on each compute node Reads the database and implements the network model / configures the network flows on the local vSwitch instance.

GRE (Generic Routing Encapsulation) Used to create virtual point-to-point links over IP (L2-in-L3) The GRE Header It has some benefits over VLANs such as being a routable protocol and not requiring e.g. 802.1Q on physical switches / physical network interfaces

Linux Network Namespaces & Neutron The ability to create logical and separate network stacks assigned to group of processes Each stack has its own routing table, firewall rules, network interfaces L3 agent creates a network namespace per virtual router and dnsmasq process Create and attach veth (virtual Ethernet interfaces) to a network namespace Connect the network namespaces to physical network interfaces via a bridge (e.g. linuxbridge or Openvswitch bridge)

A Network Packet Flow through Neutron Packet exits the VM’s ethX interface through a tap virtual Ethernet interface that connects to a Linux Bridge where iptables rules are applied. ipset optimization Connect the Linux Bridge with the OpenvSwitch br-int using a veth pair (qvb & qvo) – VLAN tagging at br-int Pass traffic to br-tun through patch interfaces Reach network node through the GRE tunnel Pass traffic to br-int (network node) through patch interfaces Deliver to virtual router and then exit though br-ex Image taken from http://pinrojas.com/2014/07/29/theres-real-magic-behind-openstack-neutron/

Neutron @ Juno Distributed Virtual Router – DVR “router_distributed” configuration under neutron.conf agent_mode= dvr_snat for network node agent_mode = dvr for compute node(s) Need for l2population mechanism Compute node(s) connects to provider network SNAT still centralized for VMs without floating IPs

Neutron @ Juno North-South (floating IP) DNAT per compute node when floatingIP is in use Default route via FIP-NS Point-to-point link between the router and the FIP namespaces Proxy-ARP for floating IPs on br-ex East-West traffic (tenant subnet-to-subnet) DVR router with the same IP and MAC address at each compute host Local DVR router maintains ARP cache of all the subnet hosts Avoid communication through the central L3 agent or ARP broadcasts

References http://docs.openstack.org/juno/install-guide/install/apt/content/section_neutron-networking.html http://pinrojas.com/2014/07/29/theres-real-magic-behind-openstack-neutron/ http://www.slideshare.net/carlbaldwin/dvr-slides http://superuser.openstack.org/articles/bridges-and-tunnels-a-drive-through-openstack-networking http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html https://www.openstack.org/summit/openstack-summit-hong-kong-2013/session- videos/presentation/openstack-neutron-modular-layer-2-plugin-deep-dive https://openstack.redhat.com/Networking_in_too_much_detail http://kimizhang.wordpress.com/2014/11/25/building-redundant-and-distributed-l3-network-in- juno

Thank You!

Appendix – Useful Commands ps aux | grep ovs - Show running OpenvSwitch processes virsh list -- Show running KVM instances virsh dumpxml instance-XXXX | less -- Show KVM instance configuration brctl show -- Show linux bridges and their connected interfaces ipset list -- View the ipset defined ethtool -S qvbXXXX - Print the interface index of the veth's peer ip link list | grep ^XX - Print the veth's peer interface ovs-vsctl list-br - Lists existing OVS bridges ovs-vsctl list-ports br-int - Lists ports per OVS bridge

Appendix – Useful Commands ovs-vsctl show - Brief overview of the database contents ovs-ofctl show br-tun -- Switch information such as flow tables and ports ovs-ofctl dump-flows br-tun -- Flows ip netns ls - List network namespaces ip netns exec $ROUTER1 ip addr - Network interfaces for a specific network namespace ip netns exec $ROUTER1 ip route - Routing table for a specific network namespace ip netns exec $ROUTER1 iptables -t nat - IP tables NAT rules for a specific network namespace