Lecture 15: IO Virtualization COSC6376 Cloud Computing Lecture 15: IO Virtualization Instructor: Weidong Shi (Larry), PhD Computer Science Department University of Houston

Outline IOV

PCI-E Endpoint Sharing 12/3/2017 8:44 PM PCI-E Endpoint Sharing © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

I/O Virtualization Intermediary 12/3/2017 8:44 PM Terminology I/O Virtualization (IOV) - the capability for a single physical I/O unit to be shared by more than one System Image I/O Virtualization Intermediary (IOVI) - software or firmware that is used to support IOV by intervening on one or more of the following: Configuration, I/O, and Memory operations from a System Image; and DMA, completion, and interrupt operations to a System Image I/O Virtualization Intermediary Virtual I/O Virtual System 1 Physical System System Image 1 Virtual System 2 System Image 2 Physical I/O © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

PCI endpoint (EP) sharing 12/3/2017 8:44 PM PCI endpoint (EP) sharing Host CPU set Host CPU set RCVE - RC Virtualization Enablers ATPT - Address Translation and Protection Table Interrupt Table SI 1 SI 2 IO VI IOVI - IO Virtualization Intermediary SI 1 SI 2 IO VI PCI Root PCI Root RCVE PCIe Switch PCIe Switch PCIe Endpoint PCIe Endpoint PCIe Endpoint PCIe IOV Endpoint PCIe IOV Endpoint PCIe IOV Endpoint EP Shared Thru Intermediary RC has no virtualization enablers One or more System Images PCIe EPs shared through IO VI IOVI is involved in all IO transactions and performs all IO Virtualization Functions, for example Multiplexes SIs’ IO queues onto a single queue in the adapter PCIe EP is not required to support any virtualization functions Natively Shared Endpoints RC has virtualization enablers One or more System Images PCIe EPs shared through IO VI Same as IOVI without ATPT PCIe IOV enabled EPs are directly shared IOVI involved in config ops Data transfers are direct © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Adapter IOV mechanisms within a single physical system 12/3/2017 8:44 PM Dedicated Adapter (No Virtualization) Adapter Shared Through Intermediary Natively Shared Adapter Graphic Depiction Intermediary Role None Virtualizes physical I/O by intervening on configuration and data transfer operations Manages assignment of Virtual Resources by intervening on configuration operations Configuration Operation Path SI direct to Adapter VI serves as proxy (SI to VI; VI to Adapter) Data Transfer Operation Path System Image 1 System Image 1 System Image 2 System Image 1 System Image 2 I/O Virtualization Intermediary I/O Virtualization Intermediary Physical System Physical System Physical System Physical I/O Physical I/O Physical I/O © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12/3/2017 8:44 PM Single Root IOV © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Single RC PCIe IOV enabled endpoint requirements 12/3/2017 8:44 PM Base PCIe 1.x System IOV Enabled PCIe System Only PCIe endpoints shall be specified for IOV enabled Endpoints Native based PCI SR OV enabled Endpoints shall be backwards compatible, in a non virtualized mode, with PCIe base 1.x SPEC Host CPU set Host CPU set SI SI 1 SI N PCIM PCI Root PCI Root PCIe Switch PCIe Switch PCI Bridge PCI Adapter PCI Adapter PCIe IOV Endpoint PCIe IOV Endpoint PCIe Switch PCIe Switch PCI-X Bridge PCIe Endpoint PCIe IOV Endpoint PCIe Endpoint PCIe IOV Endpoint PCI-X Adapter PCI-X Adapter © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Single root PCIe IOV endpoint reqs 12/3/2017 8:44 PM Single root PCIe IOV endpoint reqs Host CPU set A mechanism shall be provided to allow a VF to be associated with an SI, such that data movement operations are enabled and can be performed directly between the SI and its associated VF, without VI involvement The virtualization mechanisms defined in this specification may require a VI (such as a PCI Configuration Manager) to be involved for configuration operations performed on a VF SI 1 SI N PCIM IOV Enabled PCIe System PCI Root PCIe Switch PCIe Port PCIe IOV Endpoint Internal Routing 2. VF (Virtual Function1) 1. Physical Resources1 Configuration Management : Non-separable Resources VF (Virtual FunctionN) Physical ResourcesN Sharable Resource Pool © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

SR-IOV Physical Functions (PFs) are full PCIe devices that include the SR-IOV capabilities. Physical Functions configure and manage the SR-IOV functionality by assigning Virtual Functions. Virtual Functions (VFs) are simple PCIe functions that only process I/O. A single Ethernet port, the Physical Device, may map to many Virtual Functions that can be shared to virtualized guests. The number of Virtual Functions a device may have is limited by the device hardware.

12/3/2017 8:44 PM Multi-Root IOV © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Multi-Root PCIe IOV endpoint requirements 12/3/2017 8:44 PM Multi-Root PCIe IOV endpoint requirements The multi-root solution Shall give each RC its own Virtual Hierarchy Shall enable each switch, bridge, function, and VF to be uniquely represented in the configuration space of each RC Host CPU set PCIe Root PCI-X Device PCI-X Bridge PCI PCI Bridge PCIe IOV Endpoint PCIe Switch PCIe Physical View Host CPU set Virtual View PCIe Root PCIe Switch PCIeSwitch PCI Bridge PCIe Switch PCIe IOV Endpoint PCIe Endpoint PCI Device © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Multi-Root PCIe IOV endpoint requirements 12/3/2017 8:44 PM Multi-Root PCIe IOV endpoint requirements The multi-root solution shall Provide the same characteristics to its IOV enabled Endpoints as the single-root solution relative to separate SIs Enable use of existing PCIe 1.x or later RC Enable existing PCIe 1.x Switches, Endpoints, and PCIe to PCI/PCI-X Bridges to each be bound to a single RC Enable an IOV enabled endpoint to be shared amongst multiple RC’s using a Multi-Root Aware (MRA) PCIe switch Optional SMP Fabric Host CPU set Host CPU set 1. 1. SI 1 SI 2 SI 3 SI 3 PCIM MRA PCIe Root 2. PCIe 1.x Root MRA PCIe Switch MRA PCIe Switch 3. PCI Bridge PCI-X Adapter PCIe Endpoint PCI Adapter PCI-X Adapter 4. 3. MRA PCI-X Bridge PCI-X Adapter PCI-X Adapter MRA PCIe Switch PCI-X Adapter 4. PCIe Endpoint PCIe Endpoint © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Multi-Root PCIe IOV endpoint requirements 12/3/2017 8:44 PM Multi-Root PCIe IOV endpoint requirements The multi-root solution shall Enable a PCIe 1.x MRA bridge to PCI/PCI-X to assign each of its independent busses to a separate RC Enable multiple MRA switches to be connected together Enable an IOV enabled Endpoint to be shared by multiple SIs within one or more RC’s Enable hot-plug of physical EPs and VFs Enable a management entity (PCI Configuration Manger, PCIM) to manage the fabric Optional SMP Fabric Host CPU set Host CPU set 9. SI 1 SI 2 SI 3 SI 3 PCIM MRA PCIe Root PCIe 1.x Root MRA PCIe Switch MRA PCIe Switch 6. PCI Bridge PCI-X Adapter PCIe Endpoint PCI Adapter PCI-X Adapter 7. MRA PCI-X Bridge PCI-X Adapter PCI-X Adapter MRA PCIe Switch PCI-X Adapter 5. PCIe Endpoint PCIe Endpoint PCIe Endpoint 8. 7. 7. © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Network Device IOV 12/3/2017 8:44 PM © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Network virtualization Virtual I/O in “Native” and Guest OS Limitations of sharing I/O device in software prevent HPC applications from running in VMs Virtualization overhead Loss of “native” features Lack of SLA guarantees

Evolution of hardware IOV PCIe multi-port and multi-function solutions Single-Root (SR IOV) PCI-SIG specification Direct Hardware Access (pass-thru)

I/O virtualization and hardware SLA Over-Provisioning Bandwidth No or Limited QoS Bandwidth Over-Provisioning 2 Gbps VM1: High VM1 VM1 VM1 VM1: High VM1 VM1 VM1 VM2 2 Gbps VM2: Low VM2 VM2 Bandwidth ‘Buffer’ 2 Gbps VM3: Low VM3 VM3 VM2: Low VM2 VM2 VM2 VM3 2 Gbps VM4: High VM4 VM4 VM2 Bandwidth ‘Buffer’ VM4 2 Gbps VM5: High VM5 VM5 VM5 Time T0 T1 T2 Time T0 T1 T2 Example: 5 Virtual Machines – SLAs require 2 Gbps per VM Low Priority VM2 Receive: 5 Gbps spike Incoming traffic surge and no QoS results in SLAs being violated Traditional Solution: Over-provision bandwidth to attempt to meet SLAs Reduces VM count by 50% or more

Network virtualization with IOQoS Optimum Resource Utilization Single X3120 Replace multiple HBA and NICs with single dual port 10GbE adapter Clustering – RoCE IP Storage Data – 10GbE IOQoS™ Easy management of Prioritization and Bandwidth Allocation Direct Access for Latency sensitive (clustering) fabric VM1 VMn Application Application MPI MPI O/S O/S Exar Guest OS Driver VMxnet Driver Exar Guest OS Driver VMxnet Driver Virtual L2 Switch Exar Host Driver KVM Host Neterion X3120 X3120 configured with Multiple PCI Functions NIC 1 NIC 0 NIC 3 NIC 4 NIC 2 Integrated L2 Switch

One GPU one user Guest Guest Guest Guest VMCS1 VMCS2 VMCS3 VMCS4 Hypervisor VGA Passthrough VGA Passthrough VGA Passthrough VGA Passthrough

One GPU multiple users (emulation) GPU Emulation GPU Emulation GPU Emulation GPU Emulation GPU Emulation GPU Emu Driv GPU Emu Driv GPU Emu Driv GPU Emu Driv Guest Guest Guest Guest VMCS1 VMCS2 VMCS3 VMCS4 Hypervisor

One GPU multiple users (GPU IOV) Guest Guest Guest Guest VMCS1 VMCS2 VMCS3 VMCS4 Hypervisor Virtual function Video RAM

Network Virtualization

Reading Assignment Due: Next week