1. BESS: A Virtual Switch Tailored for NFV
Sangjin Han, Aurojit Panda, Brian Kim, Keon Jang, Joshua Reich Sai Edupuganti, Christian Maciocco, Sylvia Ratnasamy, Scott Shenker

2. Why Another Virtual Switch?
Does OpenVSwitch meet all the requirements for NFV?
Performance
OVS (~1Mpps)  OVS-DPDK (~15Mpps)
cf. Vanilla DPDK (~59Mpps/core)
Packet I/O is only half of the problem
Flexibility
Custom actions?
Stateful packet processing?
Extensibility
Must enable NFV controller evolution
Easily add support for new/niche protocols NF NF NF

3. Alternative Approach with BESS
Modular pipeline as a dataflow graph
Each module can run arbitrary code
Not limited by Match/Action semantics
Independently extensible & optimizable
Everything is programmable, not just flow tables.
You pay only for what you use.
No performance cost for unused features NF

4. BESS: Berkeley Extensible Software Switch
BESS is a programmable platform for vSwitch dataplane
Clean-slate internal architecture with NFV in mind
Highly extensible & customizable
Readily deployable with backward compatibility
… all with extreme performance:
Sub-microsecond latency
Line-rate 40Gbps with min-sized packets on two cores
(> 2x faster than other virtual switches)

5. BESS in E2 VNF VNF VNF Per-server datapath configuration
High-level NFV policy
E2 controller
BESS
E.g.,
VNF Connection
Forwarding Graph
HW NIC
HW NIC
Flow table configuration
HW Switch(es)

6. BESS Architecture Overview
BESS daemon
(running in user space)

7. BESS Architecture Overview
APP
APP OS OS VM VM
Socket APP
Native APP
APP
APP
Linux VMM
Host
Zerocopy
Container
DPDK VHost
BESS daemon
(running in user space)
Legacy
DPDK PMD

8. BESS Architecture Overview
APP
APP OS OS VM VM
Socket APP
Native APP
APP
APP
Linux VMM
Host
Zerocopy
Container
DPDK VHost
Legacy
DPDK PMD

9. BESS Architecture Overview
APP
APP OS OS
Controller VM VM
Socket APP
Native APP
APP
APP
Linux VMM
Host
Zerocopy
Container
DPDK VHost
Legacy
DPDK PMD

10. Modular Datapath Pipeline
External ports are interconnected with “modules” in a dataflow graph (like the Click modular router).
You can compose modules to implement your own datapath.
Developing a new module is easy.
Port
Port
Port
Port

11. Resource-Aware CPU Scheduler
BESS allows flexible scheduling policies for the data path.
In terms of CPU utilization and bandwidth. Examples:
Port
Port
Port
Port

12. Resource-Aware CPU Scheduler
BESS allows flexible scheduling policies for the data path.
In terms of CPU utilization and bandwidth. Examples:
“Run on core 0”
Port
Port
Port
Port
“Run on core 1”

13. Resource-Aware CPU Scheduler
BESS allows flexible scheduling policies for the data path.
In terms of CPU utilization and bandwidth. Examples:
“Higher priority”
Port
Port
Port
Port
“Lower priority”

14. Resource-Aware CPU Scheduler
BESS allows flexible scheduling policies for the data path.
In terms of CPU utilization and bandwidth. Examples:
Port
Port
Port
Port
“Limit by 1 Mpps / 5 Gbps”

15. Resource-Aware CPU Scheduler
BESS allows flexible scheduling policies for the data path.
In terms of CPU utilization and bandwidth. Examples:
“Should not consume more then 30% of CPU”
Port
Port
Port
Port

16. Control Interface
JSON-like structured messages between BESS and controller
3 ways to control the BESS datapath
Python/C APIs
Scriptable configuration language
Cisco iOS-like CLI
Everything is run-time configurable!

17. Control Interface
Port initialization
Controller / operator

18. Module initialization
Control Interface
Controller / operator
Module initialization
Dataflow setup

19. Module configuration (e.g., flow table update)
Control Interface
Module configuration (e.g., flow table update)
Controller / operator
Port stats monitoring

20. Performance?

21. Minimum Framework Overhead
Packet buffer allocation/deallocation
~10 CPU cycles per packet
CPU scheduling
~50 CPU cycles per round
Scales well with thousands of traffic classes
Dynamic per-packet metadata attributes
Zero instruction overhead for access
Optimal CPU cache-line usage

22. Performance Evaluation
OPNFV VSPERF usage models

23. 1. Phy-Phy Performance 40G line-rate
Source: Irene Liew, Anbu Murugesan – Intel IOTG/NPG

24. 1. Phy-Phy Performance
Data sources: - BESS, Vanilla DPDK, VPP: measured on a 2.6GHz Xeon E v2 machine - OVS, Linux L2/L3: Emmerich et al. “Performance Characteristics of Virtual Switching”, CloudNet OVS-DPDK: Intel ONP 2.1 Performance Test Report - mSwitch: (link bottlenecked w/ large batch 3.2GHz) Honda et al. “mSwitch: A Highly-Scalable, Modular Software Switch”, SOSR 2015

25. 2. Phy-NF-Phy Performance
VM bottleneck
Source: Irene Liew, Anbu Murugesan – Intel IOTG/NPG

26. 3. Phy-NF-NF-Phy Performance
BESS outperforms OVS-DPDK by a factor of 4-5x*
* Source: Intel ONP 2.1 Performance Test Report

27. Multi-Core/Thread Scalability

28. Round-Trip Latency RTT: 8.22us RTT: 8.82usNF NF
DPDK
DPDK
RTT: 8.22us
HW NIC
HW NIC NF NF
BESS
BESS
RTT: 8.82us
Increase of 0.60us (0.15us per BESS traverse)
DPDK
DPDK
HW NIC
HW NIC

29. Summary BESS is an ideal vSwitch platform for NFV
High performance
Sub-microsecond latency/jitter
Small packet 40Gbps throughput with only 1-2 cores
Full flexibility and extensibility
Available on GitHub:
Under BSD3 License
~30k lines in C and Python, supporting
Linux 3.x / 4.x (x86_64), DPDK 16.04
QEMU/KVM virtual machines, Docker/LXC containers