1. Better than Native: Using Virtualization to Improve Compute Node Performance
Brian Kocoloski
Jack Lange
Department of Computer Science
University of Pittsburgh
1/12/2019

2. Linux is becoming the dominant supercomputing OS …
Source:

3. … but some applications need less overhead
Lightweight Kernels (LWKs) provide low overhead access to hardware
Q: How do we provide LWKs to applications that need them, but not to those that don’t?
A: Virtualization
Applications running in a virtual environment can outperform the same applications running natively
Virtualization - We can launch LWKs inside of virtual machines (VMs) to provide isolated, low overehad environments to applications. In doing so, we improve the performance of the host OS

4. Drawbacks of Linux Memory Management OS Noise Non-technical
Biggest problem
Widely recognized as a source of overhead
OS Noise
HPC apps are tightly synchronized
Timing is a big deal
Non-technical
Noise: any events that interrupt a running program
Ex: kernel timer, task schedulers, and even general management processes such as mail clients
Overheads experienced by 1 node of a bulk synchronous parallel application are often inflicted on other nodes, as all nodes execute in lockstep
Non-technical:
Linux is not inherently an HPC OS, so it requires modification to avoid the overhead inherent in it’s design
Huge burden on OS developers that have to keep up with the ever-changing Linux code base

5. Disadvantages of Current Schemes
ZeptoOS
“Big Memory”
Memory is statically sized, allocated at boot time
Compatibility
Cray’s CNL
HugeTLBfs
Maximum of 2MB-sized memory regions available
ZeptoOS compute node Linux consists of modifications to improve the usability of the Linux kernel for HPC
ZeptoOS - “Big Memory” – preallocation of large contiguous regions of memory at boot time
Preallocated memory is static and at boot time
Compatibility – latest version based on kernel version (released in 2006)
Cray’s CNL is a runtime environment based on the Linux kernel
CNL – appears to mimic the use of hugetlbfs architecture
Limits the size of contiguous regions that can be given to applications

6. Application Completes
Our Approach
Linux
Compute Node OS
Hardware No
Linux
Compute Node OS
HPC Application
Yes
Lightweight Kernel
Application Completes
Needs LWK?
Hardware
LWKs are able to manage memory more efficiently and eliminate OS noise
After:
1. VMs allow us to launch LWKs when they are “needed” – memory available to host if not
2. Requires no modification to host OS
3. Requires no system reconfiguration
4. We are able to deliver large allocations in a way that requires no modification to the Linux kernel, in contrast to previous schemes
VMM layer
Linux
Compute Node OS
Hardware

7. Palacios OS-independent embeddable virtual machine monitor
Strip resources away from host OS
Low noise, low overhead memory management
Developed at Northwestern University, University of New Mexico, and University of Pittsburgh
Open source and freely available
Raw resources allow low overhead access for guest OS

8. Kitten Lightweight Kernel from Sandia National Labs
Moves resource management as close to application as possible
Mostly Linux-compatible user environment
Modern code-base with Linux-like organization
Open source and freely available
Like many LWKS, emphasizes efficiency over functionality – provide only what is necessary for applications – this limits noise and makes memory management easier
Familiar user-environment and code-base makes it easier for application developers

9. System Architecture

10. System Architecture
(1) Insertable as a kernel module at runtime – thus, requires no modification or reconfiguration of any kind to the Linux kernel

11. System Architecture
(1) Resource Managers – strip control of raw hardware from host OS
Raw access allows us to bypass host OS, and control two main areas (1) Memory Management, and (2) When and where Linux scheduler runs
(2) LWK/HPC on top of Palacios
(3) Finally, again, the Linux based kernel only needs to expose the module API for this approach to be feasible

12. Palacios’ Approach Memory Management OS Noise
Bypass the Linux memory management strategies completely, at run time
OS Noise
Control when the Linux scheduler is able to run
Take advantage of tickless host kernel
Linux Management
Linux Management
Palacios
Management
Offlined
Linux Management
1. Initially, we have the Linux kernel managing all of the memory in the system.
2. Hotpluggable memory interface provided by Linux allows us to “offline” – this has the effect of making the memory seem unavailable to Linux. Our approach is to offline only contiguous regions. Thus, when we take control of these offlined blocks, the resulting simplicity of guest to host memory address translations allows us to manage it with very little overhead
3. Finally, when guest OS is finished, we can return the memory to the Linux host
4. OS noise – control when by determining which VM exits will yield control to the Linux scheduler
5. Tickless host kernel – avoid frequent host timer interrupts – more on this later

13. Evaluation Two part evaluation: Evaluation is preliminary
Microbenchmarks – Stream, Selfish
Miniapplications – HPCCG, pHPCCG
Evaluation is preliminary
Currently limited to a single node running a commodity Fedora 15 kernel
Environments are not fully optimized
Microbenchmarks – used to measure memory performance and ability to control noise
Miniapplications – used to measure performance with a “real” HPC app

14. Environment Two 6-core processors and 16 GB memory
NUMA design
Kitten VM was configured with 1 GB of memory
Stream, HPCCG used OpenMP for shared memory and ran 10 times
1 GB – due to initial limitations

15. Stream
Palacios provides ~400 MB/s better memory performance on average than Linux (4.74%)
0.34 GB/s lower standard deviation on average
Stream is a microbenchmark that measures memory bandwidth

16. Selfish Detour Linux Kitten
Selfish Detour - microbenchmark that measures and detects interruptions of its execution
Fair amount of low level noise in both environments, but…
Quick: Kitten significantly less noisy – consistent, predictable

17. Selfish Detour Linux Virtualized Kitten
There is overhead – low level noise goes from roughly 7 ms to around 10, but …
…Notice host 100 HZ timer gone – thanks to tickless
With few exceptions, Palacios is indeed able to isolate the noise of the Linux host – and it doesn’t insert too much noise itself

18. HPCCG
Performs the conjugate gradient method to solve a system of linear equations represented by a sparse matrix
Workload similar to that of many HPC applications
Separate experiments to represent both CPU and memory intensive workloads
Single-precision – more CPU-intensive
Double-precision – more memory-intensive

19. HPCCG – CPU intensive
Palacios provides better performance than native Linux when running on the same NUMA node (1,2,4 cores), but slightly worse performance on 8 cores
On 8 cores, we believe that Palacios is worst because of NUMA – the Kitten VM has no understanding of the underlying NUMA layout. Linux, however, does have that information available
Palacios’ performance is most consistent which indicates that it is likely to exhibit better scalability that linux
Again, while not better across the board, Palacios was indeed better in 2 scenarios, showing that it is capable of outperforming a native Linux host

20. HPCCG – memory intensive
Only experiment in which Palacios typically performs slightly worse than native Linux – above 1 core however, results are nearly identical
However, Palacios did provide lower variance than either Linux configuration – again good for scaling

21. Future Work Extend to actual Cray hardware with a CNL host
Show definitively if this approach can work
Explore the possibility that this approach can be deployed in a cloud setting to provide virtual HPC environments on commodity clouds
Previously infeasible, due to the contention, noise, etc.
Problems we think can be solved by the same techniques used in this work

22. Conclusions
Palacios is capable of providing superior performance to native Linux
Palacios can provide a low noise environment, even when running on a noisy Linux host
While results are preliminary, they show that this approach is feasible at small scales
Superior, particularly in Stream
Feasible at small scales, and the consistency demonstrated in all environments is indicative that it is likely to scale well

23. Acknowledgments
Palacios:
Kitten: