Virtual Networking Performance Stephen Hemminger stephen.hemminger@vyatta.com Linuxcon Japan Virtualization Summit

Outline What is virtual router? Performance measurements Analysis Current work How to help

3 Tier Network Core Aggregation Access SAN Router Router Firewall Bridge Bridge Server Server Server 3 Tier Network SAN

Performance History 2005 2006 2007 2008 2009 2010 2011 1G line rate Multiqueue NIC 10G line rate 1M pps KVM Xen 3.0 Hyper-V drivers

RFC2544 Bencmark System Under Input Test Output Loss Standard for measuring router performance

Houston, we have a problem!

Physical Networks Rate limited by slowest component Adding more layers does not slow down throughput Separate resources

Virtual Network overhead Device driver Guest networking Base networking Hypervisor

Performance is a complex topic, and hard to measure. The blind man who feels a leg says the elephant is like a pillar; the one who feels the tail says the elephant is like a rope; the one who feels the trunk says the elephant is like a tree branch; the one who feels the ear says the elephant is like a hand fan; the one who feels the belly says the elephant is like a wall; and the one who feels the tusk says the elephant is like a solid pipe. A wise man explains to them: "All of you are right. The reason every one of you is telling it differently is because each one of you touched the different part of the elephant. So, actually the elephant has all the features you mentioned."

Network Baseline Intel i7 920 Intel 10G LOM 1G network Loopback – 12G bits/sec Forwarding – 1M packets/sec per core Your mileage may vary... I use Virt net every day but always felt uncomfortable that performance is being lost. Rules of thumb, today, on my hardware

Emulated Network Interface Fake PCI region Guest Hypervisor To Network Packet Buffer

Legacy (emulated) NIC In honor of Australia day wanted to graph it upside down!! Not a benchmark * different hardware * not precise * not allowed by License agreements

Virtual NIC Guest Hypervisor To Network Packet Buffer

Virtual NIC performance P7 Avoid unnecessary generality

VNIC characteristics Hyper-V Vmware vmxnet3 Xen netfront KVM virtio-net MTU 1500 9000 65521 65535 Checksum offload Y Segmentation offload NAPI LRO VLAN Multiqueue ?

Guest to Guest Throughput

VM to VM performance Relax system requirements

TCP performance

Observations Slow Legacy emulated NIC NAT Firewall connection tracking Fast Virtual NIC Hardware passthrough Bridging

Multithread benchmark

Multiple Queue HW or SW Guest VM Thread Device queue Thread Flow classfier Device queue VNIC Thread Device queue

Challenges and Opportunities Multiqueue KVM ↔ QEMU Zero copy Skb lifetime Security Batching Bufferbloat

Contributing Administrator →Tuning and Sizing Tester →Measure and Test Developer →Multiqueue and Offload