1. Virtual Networking Performance
Stephen Hemminger
Linuxcon Japan
Virtualization Summit

2. Outline What is virtual router? Performance measurements Analysis
Current work
How to help

3. 3 Tier Network Core Aggregation Access SAN Router Router Firewall
Bridge
Bridge
Server
Server
Server
3 Tier Network
SAN

5. Performance History 2005 2006 2007 2008 2009 2010 2011 1G line rate
Multiqueue NIC
10G line rate
1M pps
KVM
Xen 3.0
Hyper-V
drivers

6. RFC2544 Bencmark System Under Input Test Output Loss
Standard for measuring router performance

8. Houston, we have a problem!

9. Physical Networks Rate limited by slowest component
Adding more layers does not slow down throughput
Separate resources

10. Virtual Network overhead
Device driver
Guest
networking
Base
networking
Hypervisor

11. Performance is a complex topic, and hard to measure.
The blind man who feels a leg says the elephant is like a pillar; the one who feels the tail says the elephant is like a rope; the one who feels the trunk says the elephant is like a tree branch; the one who feels the ear says the elephant is like a hand fan; the one who feels the belly says the elephant is like a wall; and the one who feels the tusk says the elephant is like a solid pipe.
A wise man explains to them:
"All of you are right. The reason every one of you is telling it differently is because each one of you touched the different part of the elephant. So, actually the elephant has all the features you mentioned."

12. Network Baseline Intel i7 920 Intel 10G LOM 1G network
Loopback – 12G bits/sec
Forwarding – 1M packets/sec per core
Your mileage may vary...
I use Virt net every day but always felt uncomfortable that performance is being lost.
Rules of thumb, today, on my hardware

13. Emulated Network Interface
Fake PCI region
Guest
Hypervisor
To Network
Packet Buffer

14. Legacy (emulated) NIC
In honor of Australia day wanted to graph it upside down!!
Not a benchmark
* different hardware
* not precise
* not allowed by License agreements

15. Virtual NIC
Guest
Hypervisor
To Network
Packet Buffer

16. Virtual NIC performance
P7 Avoid unnecessary generality

17. VNIC characteristics Hyper-V Vmware vmxnet3 Xen netfront
KVM virtio-net
MTU
1500
9000
65521
65535
Checksum offload Y
Segmentation offload
NAPI
LRO
VLAN
Multiqueue ?

18. Guest to Guest Throughput

19. VM to VM performance
Relax system requirements

20. TCP performance

21. Observations Slow Legacy emulated NIC NAT Firewall connection tracking
Fast
Virtual NIC
Hardware passthrough
Bridging

22. Multithread benchmark

23. Multiple Queue HW or SW Guest VM Thread Device queue Thread Flow
classfier
Device queue
VNIC
Thread
Device queue

24. Challenges and Opportunities
Multiqueue
KVM ↔ QEMU
Zero copy
Skb lifetime
Security
Batching
Bufferbloat

25. Contributing Administrator →Tuning and Sizing Tester →Measure and Test
Developer →Multiqueue and Offload