1. Virtio
Keith Wiles
July 11, 2016

2. Legal Notices and Disclaimers
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. No computer system can be absolutely secure. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit Intel, the Intel logo and others are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. © 2016 Intel Corporation.
Place at the back of the deck

3. DPDK virtIO

4. Agenda
VirtIO support in DPDK
NFV Overview with DPDK and vHost

5. VIRTIO Overview
20 pt Intel Clear Bold Subhead, Date, etc.

6. DPDK -- Libraries and utilities
DPDK – VirtIO
Guest X
DPDK
Application
Guest Y
DPDK
Application
Guest Y
DPDK
Application
DPDK VirtIO Overview
DPDK contains librte_vhost source code as a library
Any application can use the rte_vhost library to help access to guest VMs
vHost is used with Open vSwitch (OVS) or standalone
A vSwitch adds VM to VM communication within the host system
VirtIO is able to be migrated when migrating a VNF
SR-IOV or other methods are more difficult
Some designs to add Crypto support to VirtIO have been suggested
VirtIO
VirtIO
VirtIO
Core N
DPDK
Application
vHost (VirtIO)
Optional vSwitch
DPDK -- Libraries and utilities
PMD 0
PMD 1
PMD 2
PMD 3
SR-IOV
SR-IOV
SR-IOV
SR-IOV
Linux Kernel
Port 0
Port 1
Port 2
Port 3

7. Life of a Packet RX Packet path:
VNF
VNF
VNF
RX Packet path:
RX Packet from port to DPKD via SR-IOV
Packet moves to the application layer and includes:
Native applications
vSwitch
VPP Fast path to VPP applications
User space Network stack to Network applications
vSwitch to VNF is one possible path for packets
TX Packet path:
VNF application sending traffic back to vSwitch
vSwitch and/or other applications sending back to DPDK
DPDK sending the packets via SR-IOV to the network device
Linux native stack path in Green using Sockets
Orange line is the standard VirtIO path to a NFV
VPP Apps
NW Apps
VirtIO 3 1
VPP
Network Stack
DPDK Apps
vSwitch
User Space 2 2
Linux Apps
DPDK 3
Linux Kernel
Native Stack
SR-IOV
Kernel Space 1
Hardware
Ports
Version: 0.2.0

8. DPDK – Data Plane development Kit
Guest X
DPDK
Application
Guest Y
DPDK
Application
Guest Y
DPDK
Application
DPDK Overview
Each application runs on a single core for best performance
If applications need to share data then the application must provide the means to communicate
DPDK does contain many types of ways to allow communication between cores lockless rings, semaphore, …
The application needs to poll the ports it wishes to receive packets of data called mbufs, RTE_MBUF
The Rx/Tx routines in DPDK manage multiple mbufs at a time for high performance
The applications also process multiple mbufs at a time
VirtIO
VirtIO
VirtIO
Core N
DPDK
Application
vHost (VirtIO)
Optional vSwitch
DPDK -- Libraries and utilities
PMD 0
PMD 1
PMD 2
PMD 3
SR-IOV
SR-IOV
SR-IOV
SR-IOV
Linux Kernel
Port 0
Port 1
Port 2
Port 3

9. DPDK – Data Plane development Kit
Guest X
DPDK
Application
Guest Y
DPDK
Application
Guest Y
DPDK
Application
DPDK Overview
The primary Rx/Tx APIs is:
static inline uint16_t
rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id,
struct rte_mbuf **rx_pkts,
const uint16_t nb_pkts);
rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id,
struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
VirtIO
VirtIO
VirtIO
Core N
DPDK
Application
vHost (VirtIO)
Optional vSwitch
DPDK -- Libraries and utilities
PMD 0
PMD 1
PMD 2
PMD 3
SR-IOV
SR-IOV
SR-IOV
SR-IOV
Linux Kernel
Port 0
Port 1
Port 2
Port 3

10. DPDK -- Libraries and utilities
DPDK – virtio example
DPDK simple startup
Guest X
DPDK
Application
Guest Y
DPDK
Application
Guest Y
DPDK
Application
int main(int argc, char ** argv) { n = rte_eal_init( argc, argv); argc -= n; argv += n; us_vhost_parse_args( argc, argv); nb_ports = rte_eth_dev_count(); /* initialize and allocate port resources */ … rte_eal_mp_remote_launch( vhost_main, NULL, CALL_MASTER); rte_vhost_driver_register(…); rte_vhost_driver_callback_register(...); return rte_vhost_driver_sessoin_start(); }
VirtIO
VirtIO
VirtIO
Core N
DPDK
Application
vHost (VirtIO)
Optional vSwitch
DPDK -- Libraries and utilities
PMD 0
PMD 1
PMD 2
PMD 3
SR-IOV
SR-IOV
SR-IOV
SR-IOV
Linux Kernel
Port 0
Port 1
Port 2
Port 3

11. DPDK -- Libraries and utilities
DPDK – Virtio example
Guest X
DPDK
Application
Guest Y
DPDK
Application
Guest Y
DPDK
Application
DPDK Overview Simple processing loop
void vhost_main(void) { /* Setup code for loop */ …. vdev = get_next_vdev(); while( core_is_running ) { if ( rx_device_ready ) drain_eth_rx(vdev); if ( tx_device_valid ) drain_virtio_tx(vdev); get_next_vdev(); }
VirtIO
VirtIO
VirtIO
Core N
DPDK
Application
vHost (VirtIO)
Optional vSwitch
DPDK -- Libraries and utilities
PMD 0
PMD 1
PMD 2
PMD 3
SR-IOV
SR-IOV
SR-IOV
SR-IOV
Linux Kernel
Port 0
Port 1
Port 2
Port 3

12. DPDK -- Libraries and utilities
DPDK – VIRTIO example
Guest X
DPDK
Application
Guest Y
DPDK
Application
Guest Y
DPDK
Application
DPDK Overview simple drain routines
void drain_eth_rx(struct vhost_dev *vdev) { n = rte_eth_rx_burst( pid, qid, pkts, MAX_BURST); if ( n ) { rte_vhost_enqueue_burst(vid, vqid, pkts, n); } void drain_virtio_tx(struct vhost_dev *vdev) { n = rte_vhost_dequeue_burst(vid, vqid, pool_id, pkts, MAX_BURST); /* Handle pkts from virtio */
VirtIO
VirtIO
VirtIO
Core N
DPDK
Application
vHost (VirtIO)
Optional vSwitch
DPDK -- Libraries and utilities
PMD 0
PMD 1
PMD 2
PMD 3
SR-IOV
SR-IOV
SR-IOV
SR-IOV
Linux Kernel
Port 0
Port 1
Port 2
Port 3

13. virtio for NFV Solutions
How to accelerate your NFV with DPDK

14. virtio-backend + vHost-user
NFV DPDK design
Focus on one design as a basic high level view:
Software Acceleration Layer is the software to hardware abstraction layer
Software Acceleration Layer makes possible additional services which can be controlled by the orchestration layer
sio-backend + vHost-user is normally in the SAL or SRL layer, but shown here to illustrate vHost in the host.
A SAL in the guest allows for the best performance selection
Direct access to hardware acceleration via SR-IOV, SOC-specific interface or other pass-though
Able to do software acceleration in the guest
SRL or HW vSwitch adds VM2VM routing or switching of packets
A SAL in the host gives scalability for non-accelerated VMs and/or native applications
Application
SAL (DPDK)
guest
Crypto
virtio
SR-IOV
virtio-backend + vHost-user
SRL(VPP)
SAL(DPDK)
host
Crypto
SR-IOV
Accelerator
HW vSwitch/Crypto
device

15. Acceleration Layer for Host
SAL: Software Acceleration Layer
Provides an abstraction between SW and HW, plus able to support multiple devices at the same time
sio-backend: backend of paravirtualized drivers
vHost-user: User space based VirtIO interface
optional for VM   host access
s-API: APIs for utilizing an AC (APIs from the AC)
g-drivers: General driver for each device type
Implemented in software or the frontend to the hardware (may be different for different acceleration functions)
hio: Hardware I/O interface
Non-virtualized, accessed only by host SAL
AC: Software/Hardware Acceleration Core (DPDK)
SRL: Software Routing Layer (OVS)
AML: Acceleration Management Layer
VM 0
VM 1
VM 2
VM 3
VM 4
sio-backend (optional vHost-user)
Software Routing Layer (SRL)
vSwitch –OVS/VPP
Acceleration Core (DPDK)
s-API or DPDK API
Software Acceleration Layer (SAL)
Acceleration Management Layer
Buffer and memory mgnt, rings/queues, ingress/egress scheduling, tasks, pipeline, …
g-drivers
SW-crypto or drivers for HW-crypto
SR-IOV(hio)

16. Further Information
Open Source Website: Documentation: or or Includes Getting Started Guides, Release Notes, Programmer’s Guide, NIC Guides, Sample App Guide, API Guide etc. Mailing Lists: The most commonly used are “dev” for development discussions and patches, and “users” for usage discussions. Webinars: search for “DPDK” Examples include “DPDK 101: Introduction to Data Plane Development Kit” and “Data Plane Development Kit – Sample Applications and New Features Deep Dive”. DPDK Events: Provides information on past and future DPDK events. Videos of presentations at previous events are available on the Past Events page FD.io (Fido) : VPP: TLDK: