1 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Scheduling I/O in Virtual Machine Monitors Diego Ongaro, Alan L. Cox, and Scott Rixner Rice University March 5, 2008

2 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Server Consolidation Virtual machine monitors enable server consolidation Consolidated domains have wildly different workloads Do current solutions actually achieve fairness? Streaming Media Server Voice-over-IP Server Application Server Web Server Database Server... Virtualized Host Guest Domain 1 Guest Domain 2 Guest Domain 3 Guest Domain 4... consolidation

3 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Xen Performance Ping latency to G8: 259.2msPing latency to G8: 6.6ms Ping additional idle guest (G8):

4 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Hypervisor – thin virtualization layer Driver Domain – hardware control Guest domains given restricted access Xen Architecture Hardware Driver Domain Xen Control Software Guest Domain User Software Guest Domain User Software Guest Domain User Software Xen Hypervisor

5 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Receiving a Network Packet time Hardware Driver Domain Xen Control Software Guest Domain User Software Guest Domain User Software Guest Domain User Software Xen Hypervisor Hardware Interrupt Virtual Interrupt Driver Domain Target Domain Scheduler Virtual Interrupt Packet Arrives

6 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Scheduling Delays Scheduler has a large effect on latency  As packets arrive, two opportunities to run any number of domains, delaying I/O  Delay proportional to the number of runnable domains time Hardware Interrupt Virtual Interrupt Driver Domain Target Domain Scheduler Virtual Interrupt Packet Arrives Other Domains

7 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Scheduler Fairness Virtual machine monitors must fairly support both computation and I/O domains  Xen supports computation domains well, but has mixed results with I/O domains The scheduler plays a significant role in I/O performance  Improvements for I/O are not obvious  Some of Xen’s current scheduler optimizations for I/O are beneficial, others are not

8 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Xen's Credit scheduler Scheduler “optimizations” for I/O Conclusions Outline

9 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Credits are assigned to each domain  Approximate the fraction of processor resources each domain will receive  Do not indicate when each domain will receive its fraction Scheduler increments/decrements credits  Periodically deducts credits from running domain  Adds credits when majority of credits in the system have been consumed Xen’s Credit Scheduler

10 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Scheduler Operation Domain states  Under – domain has credits remaining  Over – domain is over its credit allowance Domains are run in FIFO order by state  Over domains only run if no under domains  A domain may run for up to 30ms if it has enough credits After running, return to the run queue by state  Behind all other domains in the same state  Regardless of runtime or remaining credits This approach is biased against I/O O2O1 Run Queue U4U2U3 Next domain to run U1

11 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Scheduler “Optimizations” for I/O I/O Domain Requirements  Low latency  High bandwidth  Independent of other domains’ workloads “Optimizations”  Boosting idle domains Included in Xen  Ordering the run queue by credits Proposed for Xen  Tickling the scheduler Included in Xen

12 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Boosting Idle Domains Initially, scheduler had no special features for I/O domains Additional scheduling state: boost  Higher priority than under  Used when an idle domain is sent a virtual interrupt With boost O2O1U1U4U2U3 I1 U5 O2O1U1U4U2U3 I1 B1 Next domain to run Next domain to run

13 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Impact of Boost Boost improves latency for I/O domains

14 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Caveats with Boost Ineffective for I/O domains under heavy loads  Must be idle to receive boost  Must block quickly to sustain boost Effect is negated with many I/O domains O2O1B1B4B2B3 Next domain to run O2O1U1U4U2U3 vs.

15 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Ordering the Run Queue I/O domains tend to quickly block Within each state, sorts domains by credits remaining Short-running I/O domains are re-inserted near the head of the run queue

16 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Impact of Ordering the Run Queue Ordering the run queue can reduce latency for I/O domains under larger loads Surprisingly, often complements boost ConfigurationLatencyCPU Time Boost, Not Ordered 258.2ms1.2% No Boost, Ordered 39.0ms3.1% Boost, Ordered 3.5ms13.3%

17 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Tickling the Scheduler Xen tickles the scheduler after each virtual interrupt Tickling can reduce latency  Often runs virtual interrupt recipient immediately time Hardware Interrupt Virtual Interrupt Driver Domain Target Domain Scheduler Virtual Interrupt Packet Arrives Other Domains

18 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Tickling Too Soon? Tickling as virtual interrupts are sent may be too soon  Scheduler is uninformed about upcoming virtual interrupts  Gives arbitrary priority to first domain(s) to be sent a virtual interrupt Don’t tickle while sending virtual interrupts  All recipient domains are then runnable  Scheduler runs the domain with the highest priority

19 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Impact of Tickling

20 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Transcending Xen Scheduling mixed domains Native vs. VMM scheduling

21 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Mixed I/O- and compute- domains don’t underutilize resources VMM scheduler can not schedule only the I/O process Low latency for a mixed domain can not be guaranteed Mixed Domains Hypervisor Guest Domain CPU process Guest Domain CPU process I/O process Guest Domain CPU process Driver Domain Scheduler

22 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Native vs. VMM Scheduling Hypervisor Guest CPU Guest CPU I/O Guest CPU Scheduler Driver Domain ? Native Scheduler CPU process I/O processCPU process NativeVMM

23 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Summary of Results Boost reduces latency with few I/O domains  Does little harm in other cases Ordering the run queue favors short-running I/O domains  Often complementary to boost Tickling reduces latency in some cases  Often causes unfairness in bandwidth and latency I/O-intensive applications need their own domains

24 Scheduling I/O in Virtual Machine Monitors© 2008 Diego Ongaro Conclusions Scheduling has a large impact on I/O fairness  Can cause exorbitant latency  Can cause bandwidth inequity Xen’s Credit scheduler is not fair for I/O  Does well with compute domains  No notion of timeliness, which is needed for I/O Further work needs to be done to solve the mixed domains problem