KVM/ARM: The Design and Implementation of the Linux ARM Hypervisor Christoffer Dall Department of Computer Science Columbia University cdall@cs.columbia.edu.

Slides:



Advertisements
Similar presentations
Virtualization Dr. Michael L. Collard
Advertisements

Virtualization Technology
Virtualisation From the Bottom Up From storage to application.
User-Mode Linux Ken C.K. Lee
More on Processes Chapter 3. Process image _the physical representation of a process in the OS _an address space consisting of code, data and stack segments.
Chapter 6 Limited Direct Execution
Architectural Support for OS March 29, 2000 Instructor: Gary Kimura Slides courtesy of Hank Levy.
U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Emery Berger University of Massachusetts, Amherst Operating Systems CMPSCI 377 Lecture.
Disco Running Commodity Operating Systems on Scalable Multiprocessors.
1 Last Class: Introduction Operating system = interface between user & architecture Importance of OS OS history: Change is only constant User-level Applications.
KVM/ARM: The Design and Implementation of the Linux ARM Hypervisor Fall 2014 Presented By: Probir Roy.
A. Frank - P. Weisberg Operating Systems Structure of Operating Systems.
Threads CS 416: Operating Systems Design, Spring 2001 Department of Computer Science Rutgers University
Operating System Support for Virtual Machines Samuel King, George Dunlap, Peter Chen Univ of Michigan Ashish Gupta.
1 OS & Computer Architecture Modern OS Functionality (brief review) Architecture Basics Hardware Support for OS Features.
Virtualization for Cloud Computing
虛擬化技術 Virtualization and Virtual Machines
Tanenbaum 8.3 See references
Zen and the Art of Virtualization Paul Barham, et al. University of Cambridge, Microsoft Research Cambridge Published by ACM SOSP’03 Presented by Tina.
Microkernels, virtualization, exokernels Tutorial 1 – CSC469.
Protection and the Kernel: Mode, Space, and Context.
Virtualization. ABCs Special software: hypervisors or virtual machine managers Guest OS (virtual machine) sits on top of host OS (Win 7 in our case) We.
CS533 Concepts of Operating Systems Jonathan Walpole.
Operating System Support for Virtual Machines Samuel T. King, George W. Dunlap,Peter M.Chen Presented By, Rajesh 1 References [1] Virtual Machines: Supporting.
Benefits: Increased server utilization Reduced IT TCO Improved IT agility.
Xen I/O Overview. Xen is a popular open-source x86 virtual machine monitor – full-virtualization – para-virtualization para-virtualization as a more efficient.
Three fundamental concepts in computer security: Reference Monitors: An access control concept that refers to an abstract machine that mediates all accesses.
Virtualization: Not Just For Servers Hollis Blanchard PowerPC kernel hacker.
KVM/ARM: The Design and Implementation of the Linux ARM Hypervisor Christoffer Dall Department of Computer Science Columbia University
COMS E Cloud Computing and Data Center Networking Sambit Sahu
Our work on virtualization Chen Haogang, Wang Xiaolin {hchen, Institute of Network and Information Systems School of Electrical Engineering.
Lecture 3 Process Concepts. What is a Process? A process is the dynamic execution context of an executing program. Several processes may run concurrently,
CS533 Concepts of Operating Systems Jonathan Walpole.
 Virtual machine systems: simulators for multiple copies of a machine on itself.  Virtual machine (VM): the simulated machine.  Virtual machine monitor.
1 CSE451 Architectural Supports for Operating Systems Autumn 2002 Gary Kimura Lecture #2 October 2, 2002.
Introduction to virtualization
Full and Para Virtualization
Lecture 26 Virtual Machine Monitors. Virtual Machines Goal: run an guest OS over an host OS Who has done this? Why might it be useful? Examples: Vmware,
Lecture 12 Virtualization Overview 1 Dec. 1, 2015 Prof. Kyu Ho Park “Understanding Full Virtualization, Paravirtualization, and Hardware Assist”, White.
Introduction Why are virtual machines interesting?
Operating-System Structures
Protection of Processes Security and privacy of data is challenging currently. Protecting information – Not limited to hardware. – Depends on innovation.
Interrupts and Interrupt Handling David Ferry, Chris Gill CSE 522S - Advanced Operating Systems Washington University in St. Louis St. Louis, MO
E Virtual Machines Lecture 1 What is Virtualization? Scott Devine VMware, Inc.
Chapter 6 Limited Direct Execution Chien-Chung Shen CIS/UD
1 Chapter 2: Operating-System Structures Services Interface provided to users & programmers –System calls (programmer access) –User level access to system.
Virtualization Neependra Khare
Virtualization for Cloud Computing
Introduction to Virtualization
Virtualization.
Virtual Machine Monitors
Virtualization Technology
L2- Virtualization Technology
Virtualization Dr. Michael L. Collard
Operating Systems CMPSC 473
CS 6560: Operating Systems Design
Virtualization Virtualization is the creation of substitutes for real resources – abstraction of real resources Users/Applications are typically unaware.
Mechanism: Limited Direct Execution
Lecture 24 Virtual Machine Monitors
Virtualization Virtualization is the creation of substitutes for real resources – abstraction of real resources Users/Applications are typically unaware.
OS Virtualization.
Virtualization Techniques
Architectural Support for OS
Computer Security: Art and Science, 2nd Edition
CSE 451: Operating Systems Autumn 2003 Lecture 2 Architectural Support for Operating Systems Hank Levy 596 Allen Center 1.
CSE 451: Operating Systems Winter 2003 Lecture 2 Architectural Support for Operating Systems Hank Levy 412 Sieg Hall 1.
Architectural Support for OS
Xen and the Art of Virtualization
CSE 451: Operating Systems Autumn Module 24 Virtual Machine Monitors
Hypervisor A hypervisor or virtual machine monitor (VMM) is computer software, firmware or hardware that creates and runs virtual machines. A computer.
Presentation transcript:

KVM/ARM: The Design and Implementation of the Linux ARM Hypervisor Christoffer Dall Department of Computer Science Columbia University cdall@cs.columbia.edu Jason Nieh Department of Compouter Science Columbia University nieh@cs.columbia.edu 32091700 김해천

ARM ~1.2 billion ~300 million

ARM ARM Server ARM Network infrasturcture http://www.v3.co.uk/v3-uk/news/2397366/arm-aims-at-network-infrastructure-with-many-core-socs-and-open-software-stack

Virtualization Extensions Key Challenges ARM Virtualization Extensions intel VT-x != No PC-standard on ARM

Hypervisor Layering in software stack Above part HyperOne, Xen, PikeOS, OKL4, Hyper V, Vmware ESX Lower part KVM, VirtyalBox, Virtual PC, Parallels, BlueStacks

ARM Virtualization Extensions Provides virtualization in 4 key areas: CPU Virtualization Memory Virtualization Interrupt Virtualization Timer Virtualization

ARM Virtualization Extensions CPU Virtualization Hyp mode was introduced as a trap and emulate mechanism to support virtualization User User Kernel Kernel Hyp System call, page fault To reduce virtualization overhead H/w

ARM Virtualization Extensions memory virtualization Hardware support to virtualize physical memory: stage 2 Page Tables

ARM Virtualization Extensions Interrupt virtualization One distributor in a system, but each CPU core has a cpu Interface Distributor is used to configure the GIC CPU interface is used to acknowledge(ACK) and to signal End-Of- Interrupt(EOI) Interrupt can be configured to trap to either Hyp or Kernel mode Trap to kernel : avoiding the overhead of going through Hyp mode Trap to Hypervisor : hypervisor retain control, but big cost GIC v2.0 include H/W virtualization (VGIC) Virtual CPU interface, List Register VGIC PPI: Private Peripheral Interrupts SPI: Shared Peripheral Interrupts SGI: Soft Generated Interrupt GIC : Generic Interrupt Controller

ARM Virtualization Extensions Interrupt virtualization Generic Interrupt Controller : Trapping Interrupt in Hyp Mode Vm 3) Emulate Virtual Interrupt By signal Hypervisor Cumbersome & Expensive 2) trap 1) interrupt H/W

ARM Virtualization Extensions Interrupt virtualization Generic Interrupt Controller (V2.0) , Virtual GIC : Trapping Interrupt in Kernel Mode Vm Hypervisor 2) trap Good 1) interrupt H/W

ARM Virtualization Extensions Timer virtualization ARM define the Generic Timer Architecture The timers used by the hypervisor cannot be directly configured and manipulated by guest OSes. Such timer accesses from guest OS would need to trap to Hyp mode, incurring additional overhead Timer 0 CPU 0 Timer 1 CPU 1 Accessible from Hyp mode counter Timer 2 CPU 2 Timer 3 CPU 3 Virtual counter Virtual Timer 0 Virtual CPU 0 Virtual counter Virtual Timer 1 Virtual CPU 1 ARM provides ☞ Accessible from VMs Virtual counter Virtual Timer 2 Virtual CPU 2 Virtual counter Virtual Timer 3 Virtual CPU 3

Hypervisor Architecture KVM/ARM builds on KVM and leverages existing infrastructure in the Linux kernel Bare metal hypervisor(xen) vs KVM/ARM ARM platform designs are non-standard ways by different manufactures Samsung exynos, qualcomm snapdragon, Apple A series But, Linux is supported across almost all ARM platform ☞ by integrating KVM/ARM with Linux PL0 User PL1 Kernel PL2 Hyp Linux kernel KVM

Hypervisor Architecture Split-mode Virtualization Running KVM/ARM in Hyp mode implies running the Linux kernel in Hyp mode This is problematic Low-level architecture dependent code in Linux is written to work in kernel mode Running the entire kernel in Hyp mode would adversely affect native performance Kernel Kernel mode ? Kernel Hyp mode

Hypervisor Architecture Split-mode Virtualization KVM/ARM introduces split-mode virtualization It runs across different privileged CPU mode to take advantage offered by each CPU mode Two components, the lowvisor and the highvisor Lowvisor takes advantage of the H/W virtualization support available in Hyp mode Set up the correct execution context by configuration of the H/W Enforce protection and isolation between different execution context Switch from a VM execution context to the host execution, vice versa Provide a virtualization trap handler Highvisor can directly leverage existing Linux functionality Scheduler, kernel data structure, locking, memory allocation functions Kernel mode OS Kernel Hypervisor High visor Handles High level Functionality Hyp mode Low visor Handles Low level Functionality

Hypervisor Architecture Split-mode Virtualization Switching between a VM and the highvisor OS Kernel Hypervisor VM Highvisor Kernel mode Run VM Trap Trap Hyp mode Lowvisor

Hypervisor Architecture Split-mode Virtualization Switching between a VM and the highvisor OS Kernel Hypervisor VM Function call Highvisor Kernel mode Trap Trap Hyp mode Lowvisor As a result, split mode virtualization incurs a double trap cost in switching to and from the highvisor

Hypervisor Architecture CPU Virtualization Context switch register during world-switch S/W in the VM must have persistent access to same register state as S/W running on the physical CPU physical H/W state associated with the hypervisor and its host kernel is persistent across running VMs Hypervisor VM ARM Virtualized cpu Performs trap and emulate on sensitive instruction and when accessing H/W state trap Controlled by the Hypervisor

Hypervisor Architecture Memory Virtualization KVM/ARM provides memory virtualization by enabling Stage-2 translation When running in a VM Completely transparent to the VM The highvisor manages the Stage-2 translation page tables to only allow access to memory allocated for a VM Other accesses will cause stage-2 page faults which trap to the hypervisor Stage-2 translation is disabled when running in the highvisor and lowvisor

Hypervisor Architecture Memory Virtualization Configuring page tables is a high level Functionality OS Kernel Hypervisor VM Highvisor Kernel mode Hyp mode Lowvisor Configures Stage-2 Page Tables

Hypervisor Architecture Memory Virtualization LowVisor has hardware access as it runs in Hyp Mode OS Kernel Hypervisor VM Highvisor Kernel mode Hyp mode Lowvisor Enables Stage-2 Translation

Hypervisor Architecture Memory Virtualization get_user_pages() OS Kernel Hypervisor VM Highvisor Kernel mode Page fault Hyp mode Lowvisor Disables Stage-2 Translation

Hypervisor Architecture Interrupt Virtualization When running in a VM or Host & highvisor All, H/W interrupt processing is done in the host by using Linux’s existing interrupt handling func However, VM must receive notifications in the form of virtual interrupt from emulated devices KVM/ARM uses the VGIC Multicore guest Oses musts be able to send virtual IPIs to others OS Kernel VM Kernel mode High visor Hypervisor Low visor Hyp mode Trap Trap H/W

Hypervisor Architecture Timer Virtualization KVM/ARM leverage ARM’s H/W virtualization features of the generic timer Unfortunately, due to architectural limitations, the virtual timers cannot directly raise virtual interrupts, but always raise hardware interrupt, which trap to the hypervisor KVM/ARM detects when a Virtual timer expires Injects a corresponding virtual interrupt to the VM KVM/ARM performs all hardware ACK and EOI operations OS Kernel VM VM Kernel mode High visor Prepare Enable virtual timer Low visor Hyp mode Hypervisor trap timer

Experimental Setup http://www.cs.columbia.edu/~cdall/pubs/KVMARM_talk.pdf

Experimental Setup http://www.cs.columbia.edu/~cdall/pubs/KVMARM_talk.pdf

- Cost of two world switch Experimental Results Table 3 presents costs of virtualization using KVM/ARM on ARM and KVM x86 on x86 Measured in cycle units - Saving & restore VGIC state is quite expensive on ARM. - x86 provides H/W support Hypercall - Cost of two world switch Trap - Cost of switching the h/w mode from the into the cpu mode bg bg bg

Experimental Results Figure 3,4 show normalized performance for running lmbench in a VM versus Host Caused by updating the run-queue clock Lmbench : 메모리 레이턴시와 bandwidth 측정하는 밴치마크 lmbench is a suite of simple, portable, ANSI/C microbenchmarks for UNIX/POSIX. In general, it measures two key features: latency and bandwidth. lmbench is intended to give system developers insight into basic costs of key operations. Supports- KVM/ARM has less overhead than KVM x86 fork & exec Repeatedly sending an IPI Cost of KVM is higher than KVM/ARM Because this require tapping to the hypervisor on x86 but not on ARM UP: uni-processor SMP:symmetrical multi-processing

Experimental Results Figure 5,6 show normalized performance for running application workloads More mature KVM x86 system has significantly higher virtualization overheads, KVM/ARM’s split-mode virtualization design allows it to leverage ARM H/W support with comparable performance to traditional hypervisor

Experimental Results Figure 7 shows normalized power consumption of using virtualization As well as ARM, mac air’s i7 is one of Intel’s more power optimized processors Both workloads are not CPU bound & the power consumption is not significantly affected by the virtualization layer

http://www. linux-kongress. org/2010/slides/KVM-Architecture-LK2010 http://www.linux-kongress.org/2010/slides/KVM-Architecture-LK2010.pdf http://gigglehd.com/zbxe/4440629 http://www.arm.com/ http://www.cs.columbia.edu/~cdall/pubs/KVMARM_talk.pdf esc2014chhypforarmv7-140829040738.pdf kvm/arm experiences building the linux arm hypervisor

VIRQ Virtual CPU Interface ACK Virtual CPU Interface