1. Scheduling

2. Model of Process Execution Ready List Ready List Scheduler CPU Resource Manager Resource Manager Resources Preemption or voluntary yield AllocateRequest Done New Process job “Ready” “Running” “Blocked”

3. Recall Resource Manager Resource Manager Process Resource Pool ReleaseRequest Allocate Release Blocked Processes

4. Scheduler as CPU Resource Manager Scheduler Process Units of time for a time-multiplexed CPU ReleaseReady to run Dispatch Release Ready List

5. Scheduler Components Ready Process Enqueue Ready List Ready List Dispatch Context Switch Context Switch Process Descriptor Process Descriptor CPU From Other States Running Process

6. Process Context R1 R2 Rn... Status Registers Functional Unit Left Operand Right Operand Result ALU PC IR Ctl Unit

7. Context Switch CPU Process Descriptor

8. Invoking the Scheduler Need a mechanism to call the scheduler Voluntary call –Process blocks itself –Calls the scheduler Involuntary call –External force (interrupt) blocks the process –Calls the scheduler

9. Voluntary CPU Sharing yield(p i.pc, p j.pc) { memory[p i.pc] = PC; PC = memory[p j.pc]; } p i can be “automatically” determined from the processor status registers yield(*, p j.pc) { memory[p i.pc] = PC; PC = memory[p j.pc]; }

10. More on Yield yield(*, p j.pc);... yield(*, p i.pc);... yield(*, p j.pc);... p i and p j can resume one another’s execution Suppose p j is the scheduler: // p_i yields to scheduler yield(*, p j.pc); // scheduler chooses p k yield(*, p k.pc); // p k yields to scheduler yield(*, p j.pc); // scheduler chooses...

11. Voluntary Sharing Every process periodically yields to the scheduler Relies on correct process behavior –Malicious –Accidental Need a mechanism to override running process

12. Involuntary CPU Sharing Interval timer –Device to produce a periodic interrupt –Programmable period IntervalTimer() { InterruptCount--; if(InterruptCount <= 0) { InterruptRequest = TRUE; InterruptCount = K; } SetInterval(programmableValue) { K = programmableValue: InterruptCount = K; }

13. Involuntary CPU Sharing (cont) Interval timer device handler –Keeps an in-memory clock up-to-date (see Chap 4 lab exercise) –Invokes the scheduler IntervalTimerHandler() { Time++; // update the clock TimeToSchedule--; if(TimeToSchedule <= 0) { ; TimeToSchedule = TimeSlice; }

14. Contemporary Scheduling Involuntary CPU sharing – timer interrupts –Time quantum determined by interval timer – usually fixed size for every process using the system –Sometimes called the time slice length

15. Choosing a Process to Run Mechanism never changes Strategy = policy the dispatcher uses to select a process from the ready list Different policies for different requirements Ready Process Enqueue Ready List Ready List Dispatch Context Switch Context Switch Process Descriptor Process Descriptor CPU Running Process

16. Policy Considerations Policy can control/influence: –CPU utilization –Average time a process waits for service –Average amount of time to complete a job Could strive for any of: –Equitability –Favor very short or long jobs –Meet priority requirements –Meet deadlines

17. Optimal Scheduling Suppose the scheduler knows each process p i ’s service time,  p i  -- or it can estimate each  p i  : Policy can optimize on any criteria, e.g., –CPU utilization –Waiting time –Deadline To find an optimal schedule: –Have a finite, fixed # of p i –Know  p i  for each p i –Enumerate all schedules, then choose the best

18. However... The  (p i ) are almost certainly just estimates General algorithm to choose optimal schedule is O(n 2 ) Other processes may arrive while these processes are being serviced Usually, optimal schedule is only a theoretical benchmark – scheduling policies try to approximate an optimal schedule

19. Model of Process Execution Ready List Ready List Scheduler CPU Resource Manager Resource Manager Resources Preemption or voluntary yield AllocateRequest Done New Process job “Ready” “Running” “Blocked”

20. Talking About Scheduling... Let P = {p i | 0  i < n} = set of processes Let S(p i )  {running, ready, blocked} Let  (p i ) = Time process needs to be in running state (the service time) Let W(p i ) = Time p i is in ready state before first transition to running (wait time) Let T TRnd (p i ) = Time from p i first enter ready to last exit ready (turnaround time) Batch Throughput rate = inverse of avg T TRnd Timesharing response time = W(p i )

21. Simplified Model Ready List Ready List Scheduler CPU Resource Manager Resource Manager Resources AllocateRequest Done New Process job “Ready” “Running” “Blocked” Simplified, but still provide analysis result Easy to analyze performance No issue of voluntary/involuntary sharing Preemption or voluntary yield

22. Estimating CPU Utilization Ready List Ready List Scheduler CPU Done New Process System p i per second Each p i uses 1/  units of the CPU Let = the average rate at which processes are placed in the Ready List, arrival rate Let  = the average service rate  1/  = the average  (p i )

23. Estimating CPU Utilization Ready List Ready List Scheduler CPU Done New Process Let = the average rate at which processes are placed in the Ready List, arrival rate Let  = the average service rate  1/  = the average  (p i ) Let  = the fraction of the time that the CPU is expected to be busy  = # p i that arrive per unit time * avg time each spends on CPU  = * 1/  = /  Notice must have <  (i.e.,  < 1) What if  approaches 1?

24. Nonpreemptive Schedulers Ready List Ready List Scheduler CPU Done New Process Try to use the simplified scheduling model Only consider running and ready states Ignores time in blocked state: –“New process created when it enters ready state” –“Process is destroyed when it enters blocked state” –Really just looking at “small phases” of a process Blocked or preempted processes

25. First-Come-First-Served i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 T TRnd (p 0 ) =  (p 0 ) = 350 W(p 0 ) = 0 0350

26. First-Come-First-Served i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 p1p1 T TRnd (p 0 ) =  (p 0 ) = 350 T TRnd (p 1 ) = (  (p 1 ) +T TRnd (p 0 )) = 125+350 = 475 W(p 0 ) = 0 W(p 1 ) = T TRnd (p 0 ) = 350 475350

27. First-Come-First-Served i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 p1p1 p2p2 T TRnd (p 0 ) =  (p 0 ) = 350 T TRnd (p 1 ) = (  (p 1 ) +T TRnd (p 0 )) = 125+350 = 475 T TRnd (p 2 ) = (  (p 2 ) +T TRnd (p 1 )) = 475+475 = 950 W(p 0 ) = 0 W(p 1 ) = T TRnd (p 0 ) = 350 W(p 2 ) = T TRnd (p 1 ) = 475 475950

28. First-Come-First-Served i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 p1p1 p2p2 p3p3 T TRnd (p 0 ) =  (p 0 ) = 350 T TRnd (p 1 ) = (  (p 1 ) +T TRnd (p 0 )) = 125+350 = 475 T TRnd (p 2 ) = (  (p 2 ) +T TRnd (p 1 )) = 475+475 = 950 T TRnd (p 3 ) = (  (p 3 ) +T TRnd (p 2 )) = 250+950 = 1200 W(p 0 ) = 0 W(p 1 ) = T TRnd (p 0 ) = 350 W(p 2 ) = T TRnd (p 1 ) = 475 W(p 3 ) = T TRnd (p 2 ) = 950 1200950

29. First-Come-First-Served i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 p1p1 p2p2 p3p3 p4p4 T TRnd (p 0 ) =  (p 0 ) = 350 T TRnd (p 1 ) = (  (p 1 ) +T TRnd (p 0 )) = 125+350 = 475 T TRnd (p 2 ) = (  (p 2 ) +T TRnd (p 1 )) = 475+475 = 950 T TRnd (p 3 ) = (  (p 3 ) +T TRnd (p 2 )) = 250+950 = 1200 T TRnd (p 4 ) = (  (p 4 ) +T TRnd (p 3 )) = 75+1200 = 1275 W(p 0 ) = 0 W(p 1 ) = T TRnd (p 0 ) = 350 W(p 2 ) = T TRnd (p 1 ) = 475 W(p 3 ) = T TRnd (p 2 ) = 950 W(p 4 ) = T TRnd (p 3 ) = 1200 12001275

30. FCFS Average Wait Time i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 p1p1 p2p2 p3p3 p4p4 T TRnd (p 0 ) =  (p 0 ) = 350 T TRnd (p 1 ) = (  (p 1 ) +T TRnd (p 0 )) = 125+350 = 475 T TRnd (p 2 ) = (  (p 2 ) +T TRnd (p 1 )) = 475+475 = 950 T TRnd (p 3 ) = (  (p 3 ) +T TRnd (p 2 )) = 250+950 = 1200 T TRnd (p 4 ) = (  (p 4 ) +T TRnd (p 3 )) = 75+1200 = 1275 W(p 0 ) = 0 W(p 1 ) = T TRnd (p 0 ) = 350 W(p 2 ) = T TRnd (p 1 ) = 475 W(p 3 ) = T TRnd (p 2 ) = 950 W(p 4 ) = T TRnd (p 3 ) = 1200 W avg = (0+350+475+950+1200)/5 = 2974/5 = 595 127512009004753500 Easy to implement Ignores service time, etc Not a great performer

31. Predicting Wait Time in FCFS In FCFS, when a process arrives, all in ready list will be processed before this job Let  be the service rate Let L be the ready list length W avg (p) = L*1/  1/  L  Compare predicted wait with actual in earlier examples

32. Shortest Job Next i  (p i ) 0 350 1 125 2 475 3 250 4 75 p4p4 T TRnd (p 4 ) =  (p 4 ) = 75 W(p 4 ) = 0 750

33. Shortest Job Next i  (p i ) 0 350 1 125 2 475 3 250 4 75 p1p1 p4p4 T TRnd (p 1 ) =  (p 1 )+  (p 4 ) = 125+75 = 200 T TRnd (p 4 ) =  (p 4 ) = 75 W(p 1 ) = 75 W(p 4 ) = 0 200750

34. Shortest Job Next i  (p i ) 0 350 1 125 2 475 3 250 4 75 p1p1 p3p3 p4p4 T TRnd (p 1 ) =  (p 1 )+  (p 4 ) = 125+75 = 200 T TRnd (p 3 ) =  (p 3 )+  (p 1 )+  (p 4 ) = 250+125+75 = 450 T TRnd (p 4 ) =  (p 4 ) = 75 W(p 1 ) = 75 W(p 3 ) = 200 W(p 4 ) = 0 450200750

35. Shortest Job Next i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 p1p1 p3p3 p4p4 T TRnd (p 0 ) =  (p 0 )+  (p 3 )+  (p 1 )+  (p 4 ) = 350+250+125+75 = 800 T TRnd (p 1 ) =  (p 1 )+  (p 4 ) = 125+75 = 200 T TRnd (p 3 ) =  (p 3 )+  (p 1 )+  (p 4 ) = 250+125+75 = 450 T TRnd (p 4 ) =  (p 4 ) = 75 W(p 0 ) = 450 W(p 1 ) = 75 W(p 3 ) = 200 W(p 4 ) = 0 800450200750

36. Shortest Job Next i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 p1p1 p2p2 p3p3 p4p4 T TRnd (p 0 ) =  (p 0 )+  (p 3 )+  (p 1 )+  (p 4 ) = 350+250+125+75 = 800 T TRnd (p 1 ) =  (p 1 )+  (p 4 ) = 125+75 = 200 T TRnd (p 2 ) =  (p 2 )+  (p 0 )+  (p 3 )+  (p 1 )+  (p 4 ) = 475+350+250+125+75 = 1275 T TRnd (p 3 ) =  (p 3 )+  (p 1 )+  (p 4 ) = 250+125+75 = 450 T TRnd (p 4 ) =  (p 4 ) = 75 W(p 0 ) = 450 W(p 1 ) = 75 W(p 2 ) = 800 W(p 3 ) = 200 W(p 4 ) = 0 1275800450200750

37. Shortest Job Next i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 p1p1 p2p2 p3p3 p4p4 T TRnd (p 0 ) =  (p 0 )+  (p 3 )+  (p 1 )+  (p 4 ) = 350+250+125+75 = 800 T TRnd (p 1 ) =  (p 1 )+  (p 4 ) = 125+75 = 200 T TRnd (p 2 ) =  (p 2 )+  (p 0 )+  (p 3 )+  (p 1 )+  (p 4 ) = 475+350+250+125+75 = 1275 T TRnd (p 3 ) =  (p 3 )+  (p 1 )+  (p 4 ) = 250+125+75 = 450 T TRnd (p 4 ) =  (p 4 ) = 75 W(p 0 ) = 450 W(p 1 ) = 75 W(p 2 ) = 800 W(p 3 ) = 200 W(p 4 ) = 0 W avg = (450+75+800+200+0)/5 = 1525/5 = 305 1275800450200750 Minimizes wait time May starve large jobs Must know service times

38. Priority Scheduling i  (p i ) Pri 0 350 5 1 125 2 2 475 3 3 250 1 4 75 4 p0p0 p1p1 p2p2 p3p3 p4p4 T TRnd (p 0 ) =  (p 0 )+  (p 4 )+  (p 2 )+  (p 1 ) )+  (p 3 ) = 350+75+475+125+250 = 1275 T TRnd (p 1 ) =  (p 1 )+  (p 3 ) = 125+250 = 375 T TRnd (p 2 ) =  (p 2 )+  (p 1 )+  (p 3 ) = 475+125+250 = 850 T TRnd (p 3 ) =  (p 3 ) = 250 T TRnd (p 4 ) =  (p 4 )+  (p 2 )+  (p 1 )+  (p 3 ) = 75+475+125+250 = 925 W(p 0 ) = 925 W(p 1 ) = 250 W(p 2 ) = 375 W(p 3 ) = 0 W(p 4 ) = 850 W avg = (925+250+375+0+850)/5 = 2400/5 = 480 12759258503752500 Reflects importance of external use May cause starvation Can address starvation with aging

39. Deadline Scheduling i  (p i ) Deadline 0 350 575 1 125 550 2 475 1050 3 250 (none) 4 75 200 p0p0 p1p1 p2p2 p3p3 p4p4 1275 1050550200 0 Allocates service by deadline May not be feasible p0p0 p1p1 p2p2 p3p3 p4p4 p0p0 p1p1 p2p2 p3p3 p4p4 575

40. Preemptive Schedulers Ready List Ready List Scheduler CPU Preemption or voluntary yield Done New Process Highest priority process is guaranteed to be running at all times –Or at least at the beginning of a time slice Dominant form of contemporary scheduling But complex to build & analyze

41. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 W(p 0 ) = 0 050

42. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 W(p 0 ) = 0 W(p 1 ) = 50 1000 p1p1

43. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 1000 p2p2 p1p1

44. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 2001000 p3p3 p2p2 p1p1

45. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 W(p 4 ) = 200 2001000 p4p4 p3p3 p2p2 p1p1

46. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 W(p 4 ) = 200 3002001000 p0p0 p4p4 p3p3 p2p2 p1p1

47. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 T TRnd (p 4 ) =  W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 W(p 4 ) = 200 4754003002001000 p4p4 p0p0 p4p4 p3p3 p2p2 p1p1 p1p1 p2p2 p3p3

48. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 T TRnd (p 1 ) =  T TRnd (p 4 ) =  W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 W(p 4 ) = 200 4754003002001000 p4p4 p1p1 p0p0 p4p4 p3p3 p2p2 p1p1 p1p1 p2p2 p3p3 p0p0 550

49. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 T TRnd (p 1 ) =  T TRnd (p 3 ) =  T TRnd (p 4 ) =  W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 W(p 4 ) = 200 4754003002001000 p4p4 p1p1 p0p0 p4p4 p3p3 p2p2 p1p1 p1p1 p2p2 p3p3 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 550650 750850950

50. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 T TRnd (p 0 ) =  T TRnd (p 1 ) =  T TRnd (p 3 ) =  T TRnd (p 4 ) =  W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 W(p 4 ) = 200 4754003002001000 p4p4 p1p1 p0p0 p4p4 p3p3 p2p2 p1p1 p1p1 p2p2 p3p3 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p2p2 p0p0 550650 7508509501050

51. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 T TRnd (p 0 ) =  T TRnd (p 1 ) =  T TRnd (p 2 ) =  T TRnd (p 3 ) =  T TRnd (p 4 ) =  W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 W(p 4 ) = 200 4754003002001000 p4p4 p1p1 p0p0 p4p4 p3p3 p2p2 p1p1 p1p1 p2p2 p3p3 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p2p2 p0p0 p2p2 p2p2 p2p2 p2p2 550650 7508509501050115012501275

52. Round Robin (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 T TRnd (p 0 ) =  T TRnd (p 1 ) =  T TRnd (p 2 ) =  T TRnd (p 3 ) =  T TRnd (p 4 ) =  W(p 0 ) = 0 W(p 1 ) = 50 W(p 2 ) = 100 W(p 3 ) = 150 W(p 4 ) = 200 W avg = (0+50+100+150+200)/5 = 500/5 = 100 4754003002001000 Equitable Most widely-used Fits naturally with interval timer p4p4 p1p1 p0p0 p4p4 p3p3 p2p2 p1p1 p1p1 p2p2 p3p3 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p2p2 p0p0 p2p2 p2p2 p2p2 p2p2 550650 7508509501050115012501275 T TRnd _ avg = (1100+550+1275+950+475)/5 = 4350/5 = 870

53. RR with Overhead=10 (TQ=50) i  (p i ) 0 350 1 125 2 475 3 250 4 75 p0p0 T TRnd (p 0 ) =  T TRnd (p 1 ) =  T TRnd (p 2 ) =  T TRnd (p 3 ) =  T TRnd (p 4 ) =  W(p 0 ) = 0 W(p 1 ) = 60 W(p 2 ) = 120 W(p 3 ) = 180 W(p 4 ) = 240 W avg = (0+60+120+180+240)/5 = 600/5 = 120 5404803602401200 Overhead must be considered p4p4 p1p1 p0p0 p4p4 p3p3 p2p2 p1p1 p1p1 p2p2 p3p3 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p3p3 p2p2 p0p0 p2p2 p0p0 p2p2 p2p2 p2p2 p2p2 575790 910103011501270139015101535 T TRnd _ avg = (1320+660+1535+1140+565)/5 = 5220/5 = 1044 635670 790

54. Multi-Level Queues Ready List 0 Ready List 1 Ready List 2 Ready List 3 Scheduler CPU Preemption or voluntary yield Done New Process All processes at level i run before any process at level j At a level, use another policy, e.g. RR

55. Contemporary Scheduling Involuntary CPU sharing -- timer interrupts –Time quantum determined by interval timer -- usually fixed for every process using the system –Sometimes called the time slice length Priority-based process (job) selection –Select the highest priority process –Priority reflects policy With preemption Usually a variant of Multi-Level Queues

56. BSD 4.4 Scheduling Involuntary CPU Sharing Preemptive algorithms 32 Multi-Level Queues –Queues 0-7 are reserved for system functions –Queues 8-31 are for user space functions –nice influences (but does not dictate) queue level

57. Windows NT/2K Scheduling Involuntary CPU Sharing across threads Preemptive algorithms 32 Multi-Level Queues –Highest 16 levels are “real-time” –Next lower 15 are for system/user threads Range determined by process base priority –Lowest level is for the idle thread