Breaking Up is Hard to Do Security and Functionality in a Commodity Hypervisor 1 Patrick Colp†, Mihir Nanavati†, Jun Zhu‡ William Aiello†, George Coker*, Tim Deegan‡, Peter Loscocco*, Andrew Warfield† † Department of Computer Science, University of British Columbia ‡ Citrix Systems R&D * National Security Agency

2

Companies in the Cloud (all these run in EC2 or Rackspace) 3

Hypervisors are Secure 4 Hypervisor Small codebase x86 Narrow interface x86 Xen: 280 KLOC (based on the current version) Nova: 9 KLOC (microvisor) + 20 KLOC (VMM) [EuroSys’10] SecVisor: 2 KLOC [SOSP’07]Flicker: 250 LOC [EuroSys’08]

CERT Vulnerabilities 38 Xen CERT vulnerabilities 23 originate in guest VMs 2 are against the hypervisor What the heck are the other 90%? 5

6 Hypervisor Control VM (Dom0) User A’s VM User B’s VM Platform IPC Management Device Drivers Device Emulation Manage devices Create and destroy VMs Arbitrarily access memory Manage devices Create and destroy VMs Arbitrarily access memory “We are the 90%”

7 Constraint:Don’t reduce functionality, performance, or maintainability of the system Isolate services into least-privileged service VMs Make sharing between components explicit Exposure to Risk Contain scope of exploits in both space and time

SPACE 8

9 Hypervisor Control VM User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation Space

Isolation 10 Control VM Platform Device Drivers Management IPC Device Emulation Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock Builder Tools XenStore Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator

11 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Isolation

Configurable Sharing 12 User B’s Tools User A’s Tools User B’s Block User B’s Network User A’s Block User A’s Network User B’s VM User A’s VM

Configurable Sharing 13 Tools Block Network User A’s VM User B’s VM

Configurable Sharing 14 User B’s Tools User A’s Tools User B’s Block User B’s Network User A’s Block User A’s Network User B’s VM User A’s VM

15 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Isolation Configurable Sharing

Auditing 16 Create NetworkBlock Which VMs were relying on the Block component while it was compromise? VM B and VM C User A’s VM User B’s VM User C’s VM Network Block

17 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Isolation Configurable Sharing Auditing

TIME 18

19 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Containment Configurable Sharing Auditing Time

Disposable 20 Hypervisor System Boot PCI Config Services

21 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Isolation Configurable Sharing Auditing Time Disposable

Snapshots 22 VM 4-25 ms

23 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Isolation Configurable Sharing Auditing Time Disposable Timed Restarts

Stateless VMs 24 Builder User A’s VM User B’s VM Newly Created VM Snapshot Image Copy-on- Write rollback boot and initialization process request

25 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Isolation Configurable Sharing Auditing Time Disposable Timed Restarts Stateless

SPACE + TIME 26

27 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Isolation Configurable Sharing Auditing Time Disposable Timed Restarts Stateless Space + Time

Composition 28 User A’s VM User B’s VM XenStore I’ve enabled the network driver to map page 0xDEADBEEF OK B: Network can map 0xDEADBEEF I’ve enabled 0xPWND

Composition 29 User A’s VM User B’s VM XenStore-State XenStore-Logic I’ve enabled the network driver to map page 0xDEADBEEF OK B: Network can map 0xDEADBEEF I’ve enabled 0xPWNDA: Please shut me down

Composition 30 User A’s VM User B’s VM XenStore-State XenStore-Logic I’ve enabled the network driver to map page 0xDEADBEEF OK B: Network can map 0xDEADBEEF I’ve enabled 0xPWNDA: Please shut me down Monitor B Newly Created VM Snapshot Image Copy-on- Write rollback boot and initialization process request limit access

31 Hypervisor User A’s VM User B’s VM Platform Device Drivers Management IPC Device Emulation System Boot PCI Config NetworkBlock BuilderTools XenStore Emulator Space Isolation Configurable Sharing Auditing Time Disposable Timed Restarts Stateless Space + Time Composition

EVALUATION 32

Evaluation What do privileges look like now? What is the impact on the security of the system? What are the overheads? What impact does isolation have on performance? What impact do restarts have on performance? 33

Privileges 34 Privilege System Boot PCI Config BuilderToolsBlockNetworkXenStore Arbitrarily Access Memory XXXXXXX Access and Virtualize PCI devices XXXXXXX Create VMsXXXXXXX Manage VMsXXXXXXX Manage Assigned Devices XXXXXXX Privilege System Boot PCI Config BuilderToolsBlockNetworkXenStore Arbitrarily Access Memory XX Access and Virtualize PCI devices X Create VMsXX Manage VMsXXX Manage Assigned Devices XX Privilege System Boot PCI Config BuilderToolsBlockNetworkXenStore Access and Virtualize PCI devices X Create VMsXX Manage VMsXXX Manage Assigned Devices XX

Security Of the 21 vulnerabilities against the control plane, we contain all 21 TCB is reduced from the control VM’s 7.5 million lines of code (Linux) to Builder’s 13,500 (on top of Xen) 35

Memory Overhead 36 ComponentMemory System Boot128MB PCI Config128MB XenStore-Logic32MB XenStore-State32MB Block128MB Network128MB Builder64MB Tools128MB Total512MB

Isolation Performance Postmark performancewget performance 37

Restart Performance Kernel build performance 38

CONCLUSION 39

Summing it All Up Components of control VM a major source of risk Xoar isolates components in space and time – Contains exploits – Provides explicit exposure to risk Functionality, performance, and maintainability are not impacted 40