Presentation is loading. Please wait.

Presentation is loading. Please wait.

Demand-Based Coordinated Scheduling for SMP VMs

Published byGavin Barrett Modified over 6 years ago

Similar presentations

Presentation on theme: "Demand-Based Coordinated Scheduling for SMP VMs"— Presentation transcript:

1 Demand-Based Coordinated Scheduling for SMP VMs
Hwanju Kim1, Sangwook Kim2, Jinkyu Jeong1, Joonwon Lee2, and Seungryoul Maeng1 Korea Advanced Institute of Science and Technology (KAIST)1 Sungkyunkwan University2 The 18th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) Houston, Texas, March

2 Software Trends in Multi-core Era
Making the best use of HW parallelism Increasing “thread-level parallelism” Apps increasingly being multithreaded RMS apps are “emerging killer apps” “Convergence of Recognition, Mining, and Synthesis Workloads and Its Implications”, Proceedings of IEEE, 2008 SW App App App OS HW Processor Processors increasingly adding more cores

3 Software Trends in Multi-core Era
Synchronization (communication) The greatest obstacle to the performance of multithreaded workloads Thread Lock wait Barrier SW App App App OS HW Processor Spinlock Spin wait CPU

4 Software Trends in Multi-core Era
Virtualization Ubiquitous for consolidating multiple workloads “Even OSes are workloads to be handled by VMM” SW SMP VM SMP VM SMP VM App App App OS OS OS Virtual CPU (vCPU) as a software entity dictated by VMM scheduler VMM HW Processor “Synchronization-conscious coordination” is essential for VMM to improve efficiency

5 Coordinated Scheduling
Uncoordinated scheduling  A vCPU treated as an independent entity Coordinated scheduling Sibling vCPUs treated as a group (who belongs to the same VM) vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU Independent entity Time shared Time shared vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU VMM Coordinated group VMM VMM scheduler VMM scheduler pCPU pCPU pCPU pCPU pCPU pCPU pCPU pCPU vCPU Waiting Lock holder vCPU Time shared Waiting Lock waiter vCPU Running Lock waiter Uncoordinated scheduling makes inter-vCPU synchronization ineffective

6 Prior Efforts for Coordination
Coscheduling [Ousterhout82] : Synchronizing execution Relaxed coscheduling [VMware10] : Balancing execution time Stop execution for siblings to catch up vCPU execution pCPU pCPU Time pCPU Time pCPU pCPU pCPU pCPU pCPU Need for VMM scheduling based on synchronization (coordination) demands Illusion of dedicated multi-core, but CPU fragmentation Good CPU utilization & coordination, but not based on synchronization demands Balance scheduling [Sukwong11] : Balancing pCPU allocation Selective coscheduling [Weng09,11]… : Coscheduling selected vCPUs Selected vCPUs pCPU pCPU Time pCPU Time pCPU pCPU pCPU pCPU pCPU Good CPU utilization & coordination, but not based on synchronization demands Better coordination through explicit information, but relying on user or OS support

7 Overview Demand-based coordinated scheduling
Identifying synchronization demands With non-intrusive design Not compromising inter-VM fairness pCPU Demand of coscheduling for synchronization Time pCPU pCPU Demand of delayed preemption for synchronization pCPU Preemption attempt

8 Coordination Space Time and space domains
Independent scheduling decision for each domain Time When to schedule? vCPU pCPU Coordinated group Preemptive scheduling policy Coscheduling Delayed preemption Space Where to schedule? pCPU assignment policy

9 Outline Motivation Coordination in time domain
Kernel-level coordination demands User-level coordination demands Coordination in space domain Load-conscious balance scheduling Evaluation vCPU pCPU Space Time

10 Synchronization to be Coordinated
Synchronization based on “busy-waiting” Unnecessary CPU consumption by busy-waiting for a descheduled vCPU Significant performance degradation Semantic gap “OSes make liberal use of busy-waiting (e.g., spinlock) since they believe their vCPUs are dedicated”  Serious problem in kernel vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU pCPU pCPU pCPU pCPU When and where to demand synchronization? How to identify coordination demands?

11 Kernel-Level Coordination Demands
Does kernel really need coordination? Experimental analysis Multithreaded applications in the PARSEC suite Measuring “kernel time” when uncoordinated Solorun (no consolidation) A 8-vCPU VM on 8 pCPUs Corun (w/ 1 VM running streamcluster) Kernel time ratio is largely amplified by x1.3-x30  “Newly introduced kernel-level contention”

12 Kernel-Level Coordination Demands
Where is the kernel time amplified? Kernel time breakdown by functions Dominant sources TLB shootdown Lock spinning How to identify?

13 How to Identify TLB Shootdown?
Notification of TLB invalidation to a remote CPU Thread Thread Inter-processor interrupt (IPI) CPU CPU TLB TLB V->P1 V->P1 Virtual address space Busy-waiting until all corresponding TLB entries are invalidated Modify or Unmap V->P2 or V->Null V->P1 “Busy-waiting for TLB synchronization” is efficient in native systems, but not in virtualized systems if target vCPUs are not scheduled. (Even worse if TLBs are synchronized in a broadcast manner)

14 How to Identify TLB Shootdown?
TLB shootdown IPI Virtualized by VMM Used in x86-based Windows and Linux TLB shootdown IPI traffic “A TLB shootdown IPI is a signal for coordination demand!”  Co-schedule IPI-recipient vCPUs with its sender vCPU

15 How to Identify Lock Spinning?
Why excessive lock spinning? “Lock-holder preemption (LHP)” Short critical section can be unpredictably prolonged by vCPU preemption Which spinlock is problematic? vCPU pCPU Spinlock wait time breakdown 93% 82%

16 How to Identify Lock Spinning?
Futex Linux kernel support for user-level synchronization (e.g., mutex, barrier, conditional variables, etc) vCPU1 vCPU2 mutex_lock(mutex) /* critical section */ mutex_unlock(mutex) User-level contention mutex_lock(mutex) Kernel space futex_wait(mutex) { spin_lock(queue->lock) enqueue(queue, me) spin_unlock(queue->lock) schedule() /* blocked */ Kernel-level contention futex_wake(mutex) { spin_lock(queue->lock) thread=dequeue(queue) wake_up(thread) spin_unlock(queue->lock) } Preempted /* wake-up */ /* critical section */ mutex_unlock(mutex) Reschedule IPI If vCPU1 is preempted before releasing its spinlock, vCPU2 starts busy-waiting on the preempted spinlock  LHP! futex_wake(mutex) { spin_lock(queue->lock)

17 How to Identify Lock Spinning?
Why preemption-prone? Wait-queue lock Prolonged by VMM intervention VMExit APIC reg access VMEntry Remote thread wake-up Multiple VMM interventions for one IPI transmission Wait-queue unlock VMExit IPI emulation VMExit APIC reg access VMEntry VMEntry Repeated by iterative wake-up No more short critical section!  Likelihood of preemption Wait-queue lock spinning Preemption by woken-up sibling  Serious issue pCPU vCPU1 vCPU0

18 How to Identify Lock Spinning?
Generalization: “Wait-queue locks” Not limited to futex wake-up Many wake-up functions in the Linux kernel General wake-up __wake_up*() Semaphore or mutex unlock rwsem_wake(), __mutex_unlock_common_slowpath(), … “Multithreaded workloads usually communicate and synchronize on wait-queues” “A Reschedule IPI is a signal for coordination demand!” Delay preemption of an IPI-sender vCPU until a likely-held spinlock is released

19 Outline Motivation Coordination in time domain
Kernel-level coordination demands User-level coordination demands Coordination in space domain Load-conscious balance scheduling Evaluation vCPU pCPU Space Time

20 vCPU-to-pCPU Assignment
Balance scheduling [Sukwong11] Spreading sibling vCPUs on different pCPUs Increase in likelihood of coscheduling No coordination in time domain Uncoordinated scheduling Balance scheduling vCPU stacking No vCPU stacking Likelihood of coscheduling vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU < vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU pCPU pCPU pCPU pCPU pCPU pCPU pCPU pCPU

21 vCPU-to-pCPU Assignment
Balance scheduling [Sukwong11] Limitation Based on “global CPU loads are well balanced” In practice, VMs with fair CPU shares can have pCPU vCPU High scheduling latency Balance scheduling on imbalanced loads Different # of vCPUs Different TLP Multithreaded workload SMP VM vCPU vCPU vCPU vCPU SMP VM vCPU vCPU vCPU vCPU UP VM vCPU SMP VM vCPU Single-threaded workload x4 shares Inactive vCPUs TLP can be changed in a multithreaded app TLP: Thread-level parallelism

22 Proposed Scheme Load-conscious balance scheduling
Adaptive scheme based on pCPU loads When assigning a vCPU, check pCPU loads If load is balanced  Balance scheduling If load is imbalanced  Favoring underloaded pCPUs vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU Handled by coordination in time domain vCPU vCPU vCPU vCPU vCPU vCPU pCPU pCPU pCPU pCPU pCPU pCPU pCPU pCPU CPU load > Avg. CPU load  overloaded

23 Outline Motivation Coordination in time domain
Kernel-level coordination demands User-level coordination demands Coordination in space domain Load-conscious balance scheduling Evaluation

24 Evaluation Implementation Evaluation Based on Linux KVM and CFS
Effective time slice For coscheduling & delayed preemption 500us decided by sensitive analysis Performance improvement Alternative OS re-engineering

25 Non-synchronization-intensive
Evaluation SMP VM with UP VMs One 8-vCPU VM + four 1-vCPU VMs (x264) High scheduling latency Balance scheduling Performance of 8-vCPU VM LC-Balance: Load-conscious balance scheduling Resched-DP: Delayed preemption for reschedule IPI TLB-Co: Coscheduling for TLB shootdown IPI Non-synchronization-intensive Futex-intensive  5-53% improvement TLB-intensive  20-90% improvement

26 Alternative: OS Re-engineering
Virtualization-friendly re-engineering Decoupling reschedule IPI transmission from thread wake-up Delayed reschedule IPI transmission wake_up (queue) { spin_lock(queue->lock) thread=dequeue(queue) wake_up(thread) spin_unlock(queue->lock) } One 8-vCPU VM + four 1-vCPU VMs (x264) Reschedule IPI Modified wake_up func Using per-cpu bitmap Applied to futex_wakeup & futex_requeue Delayed reschedule IPI is virtualization-friendly to resolve LHP problems

27 Conclusions & Future Work
Demand-based coordinated scheduling IPI as an effective signal for coordination pCPU assignment conscious of dynamic CPU loads Limitation Cannot cover ALL types of synchronization demands Kernel spinlock contention w/o VMM intervention Future work Cooperation with HW (e.g., PLE) & paravirt Barrier or lock Address space

28 Thank You! Questions and comments Contacts hjukim@calab.kaist.ac.kr

29 Extra Slides

30 User-Level Coordination Demands
Coscheduling-friendly workloads SPMD, bulk-synchronous, etc. Busy-waiting synchronization “Spin-then-block” More blocking operations when uncoordinated Coscheduling (balanced execution) Uncoordinated (skewed execution) Uncoordinated (largely skewed execution) Z Thread1 Thread2 Thread3 Thread4 Z Thread1 Thread2 Thread3 Thread4 Z Thread1 Thread2 Thread3 Thread4 Barrier Z Z Z Z Wake up Z Wake up Z Wake up Spin Block Wake up Barrier Additional barrier Wake up Z Z Z

31 User-Level Coordination Demands
Coscheduling Avoiding more expensive blocking in a VM VMExits for CPU yielding and wake-up Halt (HLT) and Reschedule IPI When to coschedule? User-level synchronization involves reschedule IPIs Reschedule IPI traffic of streamcluster Barriers “A Reschedule IPI is a signal for coordination demand!” Co-schedule IPI-recipient vCPUs with a sender vCPU Providing a knob to selectively enable this coscheduling for coscheduling-friendly VMs

32 Urgent vCPU First (UVF) Scheduling
1. Preemptively scheduled if fairness is kept 2. Protected from preemption once scheduled During “Urgent time slice (utslice)” vCPU : urgent vCPU Urgent queue Runqueue vCPU vCPU vCPU vCPU Protected from preemption pCPU (or reschedule IPI if needed) TLB shootdown IPI Reschedule IPI FIFO order Proportional shares order Wait queue Coscheduled vCPU vCPU vCPU vCPU vCPU vCPU pCPU If inter-VM fairness is kept

33 Proposed Scheme Load-conscious balance scheduling
Adaptive scheme based on pCPU loads Balanced loads  Balance scheduling Imbalanced loads  Favoring underloaded pCPUs vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU pCPU pCPU pCPU pCPU pCPU pCPU pCPU pCPU Example Candidate pCPU set (Scheduler assigns a lowest-loaded pCPU in this set) vCPU vCPU vCPU = {pCPU0, pCPU1, pCPU2, pCPU3} Wait queue vCPU pCPU3 is overloaded (i.e., CPU load > Avg. CPU load) Handled by coordination in time domain (UVF scheduling) vCPU vCPU vCPU pCPU0 pCPU1 pCPU2 pCPU3

34 Evaluation Urgent time slice (utslice) 1. Utslice for reducing LHP
2. Utslice for quickly serving multiple urgent vCPUs Workloads: A futex-intensive workload in one VM + dedup in another VM as a preempting VM >300us utslice 2x-3.8x LHP reduction Remaining LHPs occur during local wake-up or before reschedule IPI transmission  Unlikely lead to lock contention

35 Evaluation Urgent time slice (utslice) 1. utslice for reducing LHP
2. utslice for quickly serving multiple urgent vCPUs ~11% degradation Workloads: 3 VMs, each of which runs vips (vips - TLB-IPI-intensive application) As utslice increases, TLB shootdown cycles increase 500usec is an appropriate utslice for both LHP reduction and multiple urgent vCPUs

36 Evaluation Urgent allowance Improving overall efficiency with fairness
Workloads: vips (TLB-IPI-intensive) VM + two facesim VMs No performance drop Efficient TLB synchronization

37 Evaluation Impact of kernel-level coordination
One 8-vCPU VM + four 1-vCPU VMs (x264) Unfair contention Balance scheduling Performance of 1-vCPU VM LC-Balance: Load-conscious balance scheduling Resched-DP: Delayed preemption for reschedule IPI TLB-Co: Coscheduling for TLB shootdown IPI Balance scheduling  Up to 26% degradation

38 Evaluation: Two SMP VMs
w/ dedup a: baseline b: balance c: LC-balance d: LC-balance+Resched-DP e: LC-balance+Resched-DP+TLB-Co w/ freqmine

39 Reduction in Pause-loop VMExits (%)
Evaluation Effectiveness on HW-assisted feature CPU feature to reduce the amount of busy-waiting VMExit in response to excessive busy-waiting Intel Pause-Loop-Exiting (PLE), AMD Pause Filter Inevitable cost of some busy-waiting and VMExit streamcluster (futex-intensive) ferret (TLB-IPI-intensive) LHP PAUSE PAUSE Threshold … PAUSE VMExit Yielding Apps Streamcluster facesim ferret vips Reduction in Pause-loop VMExits (%) 44.5 97.7 74.0 37.9

40 Evaluation Coscheduling-friendly user-level workload Streamcluster
Spin-then-block barrier intensive workload Normalized execution time (corunning w/ bodytrack) Barrier breakdown Additional barriers More performance improvement as the time of spin-waiting increases Blocking: 38% Reschedule IPIs (3 VMExits): 21% Additional (departure) barriers: 29% Resched-Co: Coscheduling for rescheudle IPI

Download ppt "Demand-Based Coordinated Scheduling for SMP VMs"

Similar presentations

KAIST Computer Architecture Lab. The Effect of Multi-core on HPC Applications in Virtualized Systems Jaeung Han¹, Jeongseob Ahn¹, Changdae Kim¹, Youngjin.

KAIST Computer Architecture Lab. The Effect of Multi-core on HPC Applications in Virtualized Systems Jaeung Han¹, Jeongseob Ahn¹, Changdae Kim¹, Youngjin.
Abdulrahman Idlbi COE, KFUPM Jan. 17, 2010. Past Schedulers: 1.2 & 2.2 1.2: circular queue with round-robin policy. Simple and minimal. Not focused on.

Abdulrahman Idlbi COE, KFUPM Jan. 17, Past Schedulers: 1.2 & : circular queue with round-robin policy. Simple and minimal. Not focused on.
Bilgisayar Mühendisliği Bölümü GYTE - Bilgisayar Mühendisliği Bölümü Multithreading the SunOS Kernel J. R. Eykholt, S. R. Kleiman, S. Barton, R. Faulkner,

Bilgisayar Mühendisliği Bölümü GYTE - Bilgisayar Mühendisliği Bölümü Multithreading the SunOS Kernel J. R. Eykholt, S. R. Kleiman, S. Barton, R. Faulkner,
Hardware Support for Spin Management in Overcommitted Virtual Machines Philip Wells Koushik Chakraborty Gurindar Sohi {pwells, kchak,

Hardware Support for Spin Management in Overcommitted Virtual Machines Philip Wells Koushik Chakraborty Gurindar Sohi {pwells, kchak,
Chapter 4: Threads. Overview Multithreading Models Threading Issues Pthreads Windows XP Threads.

Chapter 4: Threads. Overview Multithreading Models Threading Issues Pthreads Windows XP Threads.
1 CS318 Project #3 Preemptive Kernel. 2 Continuing from Project 2 Project 2 involved: Context Switch Stack Manipulation Saving State Moving between threads,

1 CS318 Project #3 Preemptive Kernel. 2 Continuing from Project 2 Project 2 involved: Context Switch Stack Manipulation Saving State Moving between threads,
Scheduler Activations Effective Kernel Support for the User-Level Management of Parallelism.

Scheduler Activations Effective Kernel Support for the User-Level Management of Parallelism.
3.5 Interprocess Communication Many operating systems provide mechanisms for interprocess communication (IPC) –Processes must communicate with one another.

3.5 Interprocess Communication Many operating systems provide mechanisms for interprocess communication (IPC) –Processes must communicate with one another.
3.5 Interprocess Communication

3.5 Interprocess Communication
Chapter 11 Operating Systems

Chapter 11 Operating Systems
Informationsteknologi Tuesday, October 9, 2007Computer Systems/Operating Systems - Class 141 Today’s class Scheduling.

Informationsteknologi Tuesday, October 9, 2007Computer Systems/Operating Systems - Class 141 Today’s class Scheduling.
Scheduler Activations Jeff Chase. Threads in a Process Threads are useful at user-level – Parallelism, hide I/O latency, interactivity Option A (early.

Scheduler Activations Jeff Chase. Threads in a Process Threads are useful at user-level – Parallelism, hide I/O latency, interactivity Option A (early.
Synchronization and Scheduling in Multiprocessor Operating Systems

Synchronization and Scheduling in Multiprocessor Operating Systems
Multiple Processor Systems. Multiprocessor Systems Continuous need for faster and powerful computers –shared memory model ( access nsec) –message passing.

Multiple Processor Systems. Multiprocessor Systems Continuous need for faster and powerful computers –shared memory model ( access nsec) –message passing.
Object Oriented Analysis & Design SDL Threads. Contents 2  Processes  Thread Concepts  Creating threads  Critical sections  Synchronizing threads.

Object Oriented Analysis & Design SDL Threads. Contents 2  Processes  Thread Concepts  Creating threads  Critical sections  Synchronizing threads.
1 Scheduling Processes. 2 Processes Each process has state, that includes its text and data, procedure call stack, etc. This state resides in memory.

1 Scheduling Processes. 2 Processes Each process has state, that includes its text and data, procedure call stack, etc. This state resides in memory.
Cosc 4740 Chapter 6, Part 3 Process Synchronization.

Cosc 4740 Chapter 6, Part 3 Process Synchronization.
Fast Multi-Threading on Shared Memory Multi-Processors Joseph Cordina B.Sc. Computer Science and Physics Year IV.

Fast Multi-Threading on Shared Memory Multi-Processors Joseph Cordina B.Sc. Computer Science and Physics Year IV.
Scheduling Basic scheduling policies, for OS schedulers (threads, tasks, processes) or thread library schedulers Review of Context Switching overheads.

Scheduling Basic scheduling policies, for OS schedulers (threads, tasks, processes) or thread library schedulers Review of Context Switching overheads.
Operating Systems CSE 411 Multi-processor Operating Systems Multi-processor Operating Systems Dec. 8 2006 - Lecture 30 Instructor: Bhuvan Urgaonkar.

Operating Systems CSE 411 Multi-processor Operating Systems Multi-processor Operating Systems Dec Lecture 30 Instructor: Bhuvan Urgaonkar.

Similar presentations

Ads by Google