1. Kernel-based Virtual Machine (KVM) Memory Management Unit (MMU) Virtualization Mohammad H. Mofrad February 11, 2016 1hasanzadeh@cs.pitt.edu

2. Contents Kernel-based Virtual Machine Linux memory management unit memory management unit virtualization 2

3. KVM as a Virtual Machine Monitor (KVM) 3 Hardware Linux Kernel Kvm.ko /dev/kvm Virtual MachineHost Applications qemu Architecture of the Kernel-based Virtual Machine (KVM), Jan Kiszka, Siemens AG open("/dev/kvm") ioctl(KVM_CREATE_VM) ioctl(KVM_CREATE_VCPU) ioctl(KVM_MEMORY_REGION) for (;;) { ioctl(KVM_RUN) switch (exit_reason) { case KVM_EXIT_IO: /*... */ case KVM_EXIT_HLT: /*... */ }

4. Linux Memory Management Unit (MMU) A Memory Management Unit (MMU) translates physical memory addresses to virtual address used by the operating system. It requests a page fault interrupt MMU uses a Translation Lookaside Buffer (TLB) to improve virtual address translation speed. 4Source: The Linux Kernel’s Memory Management Unit API, William Gatliff Host Physical Memory ------------- Page #1 ------------- Page #2 ------------- … MMU TLB ----------------------- | CPU HVA HPA Host Virtual Address (HVA) Host Physical Address (HPA) Physical Machine

5. MMU in Virtualization 5 MMU TLB ----------------------- | CPU GVA GPA Guest Physical Memory ------------- Page #1 ------------- Page #2 ------------- … MMU Virtualization Host Physical Memory | | | | | | | … Host Virtual Address (HVA) Host Physical Address (HPA) Guest Virtual Address (GVA) Guest Physical Address (GPA) Virtual Machine Physical Machine

6. Memory virtualization problems 3 levels of indirection But MMU can translate only one But MMU virtualization translates GVA to HPA. 6Virtualization in Linux KVM + QEMU GVAGPAHVAHPA >>

7. Memory Virtualization Shadow paging Hardware assisted virtualization MMU paravitualization 7

8. Shadow Paging 8 A|B|C|D|E|F|G|H|I|J| |… 1|2|3|4| |… 1|2|3|4|5|6|7|8|9|10| |… 1|2|3|4|5|6| |… GVA GPA HVA HPA KVM\Qemu GPA >> HPA VM GVA >> GPA Shadow page table Guest process page table ----------------------- | Shadow page table ----------------------- | Page fault >> VM EXIT

9. Shadow Paging pros and cons + + Software solution + + No need to change guest OS + + No special hardware is required - - Every guest page needs a shadow version -- Shadow page table must be consistent with guest and host 9Virtualization in Linux KVM + QEMU

10. Hardware Assisted Virtualization (NTP/EPT ) Eliminate the need for shadow paging (software approach) Provide architectural support Nested Page Table (NPT) by AMD Extended Page Table (EPT) by Intel 10Address Translation for Virtual Machines

11. Hardware Assisted Virtualization (NTP/EPT ) 11 A|B|C|D|E|F|G|H|I|J| |… 1|2|3|4| |… 1|2|3|4|5|6|7|8|9|10| |… 1|2|3|4|5|6| |… GVA GPA HVA HPA KVM/Qemu GPA >> HPA VM GVA >> GPA NTP/EPT table Guest process page table ----------------------- | N/ETP page table ----------------------- | Page fault >> NTP/EPT violation

12. NTP/EPT Pros and Cons + + OS independent + + No need to change guest OS + + NTP/EPT table is proportional to guest memory - - Need hardware support -- Considerable overhead for TLB misses -- More memory access + longer latency 12Virtualization in Linux KVM + QEMU

13. MMU Paravirtualization Paravirtualization involves modifying OS kernel to replace nonvirtualize instructions with virtualization hypercalls. MMU paravirtualization is one of the original innovations of Xen VMM. So, guest is aware of how to map GVA to HPA!!! 13http://wiki.xenproject.org/wiki/X86_Paravirtualised_Memory_Management

14. MMU Paravirtualization 14http://wiki.xenproject.org/wiki/X86_Paravirtualised_Memory_Management A|B|C|D|E|F|G|H|I|J| |… 1|2|3|4| |… GVA HPA Xen direct paging GVA >>HPA G2P Guest process page table ----------------------- |. | Page fault >> NTP/EPT violation

15. MMU Paravirtualization Pros and Cons + + Near native solution + + Low memory overhead while taking full advantage of hardware - - Compatibility cost [modifying the guest kernel] -- All updates/changes need to be validated by VMM -- More memory access + longer latency 15kvm: the Linux virtual machine monitor

16. Useful References [1]Kivity, Avi, et al. "kvm: the Linux virtual machine monitor." Proceedings of the Linux symposium. Vol. 1. 2007. [2 ] Bae, Chang S., John R. Lange, and Peter A. Dinda. "Enhancing virtualized application performance through dynamic adaptive paging mode selection.“ Proceedings of the 8th ACM international conference on Autonomic computing. ACM, 2011. [3]Wang, Xiaolin, et al. "Selective hardware/software memory virtualization.“ ACM SIGPLAN Notices 46.7, 217-226, 2011. 16