7/2/20161 Re-architecting VMMs for Multicore Systems: The Sidecore Approach Presented by: Sanjay Kumar PhD Candidate, Georgia Institute of Technology Co-Authors: Himanshu Raj, Prof. Karsten Schwan, Dr. Ivan Ganev Georgia Institute of Technology

7/2/20162 Background Multicore and Manycore Systems: –Increases parallelism significantly even in small scale systems –What can be done with so many cores –Creates new opportunities for novel programming models Virtualization: –Provides isolation, consolidation, manageability, containment –Becoming popular even in small scale systems –Multi-core systems will create design challenges for VMMs

7/2/20163 Virtual Machine Monitor(s) Every core executes potentially all VMM code –How to reduce the state transition overheads How to best virtualize/utilize the specialized heterogeneous cores –These cores may run their virtualization code themselves –They may run the specialized functionality

7/2/20164 Alternative VMM design Restructure the VMM as multiple components Components responsible for certain VMM functionality Different cores can run different components –Alternatively moved to hardware Sidecore: Abstraction of service cores providing services to guest VMs

7/2/20165 Advantage Sidecore Creates ‘specialized’ cores –Improved performance because of reduced processor state changes Useful for specialized heterogeneous cores –They can be abstracted as service cores Creates an alternative model for VMM design for future multi-core systems

7/2/20166 Disadvantages Some cores may be underutilized –Not an issue in large scale multi-core environments –Can be alleviated by dynamically switching Sidecore functionality on/off Programming challenges –VMM/OS will need to be modified –Synchronization issues in using Sidecores

7/2/20167 Rest of the Talk Sample usage scenarios for Sidecores –Homogeneous cores as Sidecores Improved VM-VMM communication Interrupt virtualization in SV-NIC –Heterogeneous cores as Sidecores Ongoing work Network processors, management processors

7/2/20168 Improving VM-VMM Communication in VT enabled VMs Xen Service VM VT enabled VM user kernel C C S V V VMExit/VMEntry V = CPU in VM mode C = CPU in VMM mode S = CPU as sidecore Sidecall VM’s privileged operations cause VMExit Currently VMExits are very costly operations Alternatively VM requests Sidecore to execute privileged operation via sidecall Sidecore already in VMM mode Avoids VMExit

7/2/20169 Page Table Updates VM’s attempt to modify PTEs causes VMExit Alternatively VM requests Sidecore to update its PTEs via sidecall Sidecore updates VM’s PTEs and corresponding shadow PTEs Avoids VMExit during PTE update

7/2/ VMExit vs. Sidecall Latency Remote TLB shootdowns cause VMExit VMExit latency will reduce –So will the Sidecall latency Recent MMU virtualization advances –The PT management moved to MMU

7/2/ LMBench Result

7/2/ Interrupt Virtualization in SV-NIC SV-NIC PCI Communication Network Controller Domain VIFs Mgmt VIF0VIF1 Mgmt Host Dom0 Guest 0 Guest 1 Hypervisor

7/2/ Transmit Operation –Polling by SV-NIC Receive Operation –SV-NIC signals host via PCI interrupt –Interrupt virtualization via identifier –Master PCI interrupt by 8 bit identifier Interrupt Virtualization Details

7/2/ Interrupt Virtualization using Sidecore Limitations of Interrupt Identifier –May result in redundant signalling for large number of VMs SV-NIC + Sidecore –Signal virtualization via dedicated host core No interrupts from SV-NIC Polling by Sidecore

7/2/ Experimental Results – Latency

7/2/ Ongoing and Future Work Dynamically deploy Sidecore functionality on cores –Sidecores to VMs ratio –Effects of Sidecore allocation on overall VM performance Use heterogeneous cores –Management processors, Network processors, GPUs Stub-domains –Lightweight domains running specialized code on Sidecores

7/2/ Questions??