GPU Virtualization on VMware’s Hosted I/O Architecture Micah Dowty Jeremy Sugerman USENIX WIOV

Contents GPUs are hard But GPU virtualization is worth the trouble How to virtualize a GPU? VMware’s virtual GPU Conclusions In the paper: – Details on our implementation – Benchmarks, analysis 2

What is a GPU, anyway? Video playback, 2D graphics, drawing triangles and rectangles and lines... Computation. 3

How much computation? Source: AnandTech review of NVidia GT200 Intel Core 2 Duo: 291 million transistors NVIDIA GeForce GTX 280 : 1.4 billion transistors 4

Programmable 3D Pipeline Vertex DataVertex Pipeline Position Material Texture coordinate Rasterization Plot triangles, lines Interpolate vertices Pixel PipelineFramebuffer Texture Data State Pixel Shader State Vertex Shader (State of the art circa ) 5

Unique challenges API – Not quite read(), write(), select()... – Multiple competing APIs – Hundreds of entry points Programmable – Every GPU driver is also a compiler – Each API includes a language spec 6

Unique challenges Hardware specs – Diverse, changes frequently – Closely guarded secret * – Speed vs. portability Hardware state – Up to gigabytes of data – Highly device-specific format – In-progress DMA and computation GPU App GPU Driver App OpenGLDirect3DCompute 7 * With a few notable exceptions.

What are GPUs good for? Desktop Apps – Entertainment – CAD – Multimedia – Productivity Desktop GUIs – Quartz Extreme – Vista Aero – Compiz 8

GPUs in the Data Center Server-hosted Desktops GPGPU 9

API Remoting GPU GPU Driver OpenGL / Direct3D Hardware Kernel HostGuest API User-levelRPC Endpoint App APIOpenGL / Direct3D Redirector

Device Emulation GPU GPU Driver OpenGL / Direct3D Hardware Kernel HostGuest API User-level Rendering Backend Virtual GPU Virtual GPU Driver App Virtual HW Kernel APIOpenGL / Direct3D Shader / State Translator Resource Management GPU Emulator Shared System Memory

Virtual Machine Fixed pass-through OpenGL / Direct3D / Compute App GPU Driver App API Pass-through GPU Physical GPU PCIIRQMMIO VT-d DMA

Mediated pass-through Physical GPU GPU Resource Manager Virtual Machine OpenGL / Direct3D / Compute App GPU Driver App API Emulation Pass-through GPU Virtual Machine OpenGL / Direct3D / Compute App GPU Driver App API Emulation Pass-through GPU

GPU Virtualization Taxonomy Front-end Back-end Fixed Pass-through 1:1 Fixed Pass-through 1:1 Mediated Pass-through 1:N Mediated Pass-through 1:N Hybrid (Driver VM) Hybrid (Driver VM) API RemotingDevice Emulation 14

VMware’s Virtual GPU Compatibility – Any physical GPU – Any guest driver stack – Adjustable capability exposure – No direct access to GPU memory Efficiency – Flexible guest memory management – Few copies – Asynchronous rendering 15 GPU GPU Driver OpenGL / Direct3D Rendering Backend VMware SVGA II Virtual GPU Driver App OpenGL / Direct3D Shader / State Translator Resource Management Device Emulation

VMware SVGA II 16

Virtual Graphics Stack GPU GPU API / Driver Host Guest SVGA FIFO / Registers VMware SVGA Driver DMA Engine MKS / HostOps Dispatch SVGA Device Guest Mem Shader Program Translator Guest VRAM SVGA GMR App 3D Rendering Surface Abstraction State Translator 3D Drawing Path 2D Compositing 2D 3D Video 17

Evaluation Applications Microbenchmarks VMware Fusion 2.0, VMware Workstation 6.5, Parallels Desktop 3.0, SwiftShader Mac Pro, 8-core 2.8 GHz ATI Radeon HD

Application Benchmarks 19

Summary GPU Virtualization is an important problem Room for improvement in implementation completeness and performance... But we can already run interactive apps that could never be virtualized before Virtual GPU preserves portability + isolation 20

Future Work Pass-through techniques – Fixed and Mediated – Can be complementary to Virtual GPU Continued improvements – Performance and functionality – At all layers of driver stack Virtualization-aware GPU benchmarks 21

Questions? 22