1. Resource Allocation & Scheduling
CS 5204 Operating Systems
Resource Allocation & Scheduling
Godmar Back

2. Resource Allocation and Scheduling

3. Resource Allocation & Scheduling
Resource management is primary OS function
Involves resource allocation & scheduling
Who gets to use what resource and for how long
Example resources:
CPU time
Disk bandwidth
Network bandwidth
RAM
Disk space
Processes are the principals that use resources
often on behalf of users
CS 5204 Fall 2015

4. Preemptible vs Nonpreemptible Resources
Once allocated, can’t easily ask for them back – must wait until process returns them (or exits)
Examples: Locks, Disk Space, Control of terminal
Preemptible resources:
Can be taken away (“preempted”) and returned without the process noticing it
Examples: CPU, Memory
CS 5204 Fall 2015

5. Physical vs Virtual Memory
Classification of a resource as preemptible depends on price one is willing to pay to preempt it
Can theoretically preempt most resources via copying & indirection
Virtual Memory: mechanism to make physical memory preemptible
Take away by swapping to disk, return by reading from disk (possibly swapping out others)
Not always tolerable
resident portions of kernel
Pintos kernel stack pages
CS 5204 Fall 2015

6. Space Sharing vs Time Sharing
Space Sharing: Allocation (“how much?”)
Use if resource can be split (multiple CPUs, memory, etc.)
Use if resource is non-preemptible
Time Sharing: Scheduling (“how long?”)
Use if resource can’t be split
Use if resource is easily preemptible
CS 5204 Fall 2015

7. CPU vs. Other Resources
CPU is not the only resource that needs to be scheduled
Overall system performance depends on efficient use of all resources
Resource can be in use (busy) or be unused (idle)
Duty cycle: portion of time busy
Consider I/O device: busy after receiving I/O request – if CPU scheduler delays process that will issue I/O request, I/O device is underutilized
Ideal: want to keep all devices busy
CS 5204 Fall 2015

8. Per-process perspective
Process alternates between CPU bursts & I/O bursts
I/O Bound Process
CPU Bound Process
CPU
I/O
CS 5204 Fall 2015

9. Global perspective If these were executed on the same CPU: Waiting CPU
I/O Bound Process
CPU Bound Process
Waiting
CPU
I/O
CS 5204 Fall 2015

10. CPU Scheduling
Part I

11. CPU Scheduling Terminology
A job (sometimes called a task, or a job instance)
Activity that’s scheduled: process or part of a process
Arrival time: time when job arrives
Start time: time when job actually starts
Finish time: time when job is done
Completion time (aka Turn-around time)
Finish time – Arrival time
Response time
Time when user sees response – Arrival time
Execution time (aka cost): time a job needs to execute
Arrival Time
Start Time
Finish Time
waiting
CPU burst
I/O
waiting
CPU
Response Time
Completion Time
CS 5204 Fall 2015

12. CPU Scheduling Terminology (2)
Waiting time = time when job was ready-to-run
didn’t run because CPU scheduler picked another job
Blocked time = time when job was blocked
while I/O device is in use
Completion time
Execution time + Waiting time + Blocked time
CS 5204 Fall 2015

13. Static vs Dynamic Scheduling
All jobs, their arrival & execution times are known in advance, create a schedule, execute it
Used in statically configured systems, such as embedded real-time systems
Dynamic or Online Scheduling
Jobs are not known in advance, scheduler must make online decision whenever jobs arrives or leaves
Execution time may or may not be known
Behavior can be modeled by making assumptions about nature of arrival process
CS 5204 Fall 2015

14. Scheduling Algorithms vs Scheduler Implementations
Scheduling algorithms’ properties are (usually) analyzed under static assumptions first; then adapted for dynamic scenarios
Algorithms often consider only an abstract notion of (CPU) “jobs”, but a dynamic scheduler must map that to processes with alternating - and repeating - CPU and IO bursts
Often applies static algorithm to current ready queue
Algorithms often assume length of job/CPU burst is known, but real scheduler must estimate expected execution cost (or make assumptions)
CS 5204 Fall 2015

15. Preemptive vs Nonpreemptive Scheduling
Q.: when is scheduler asked to pick a thread from ready queue?
Nonpreemptive:
Only when RUNNING BLOCKED transition
Or RUNNING  EXIT
Or voluntary yield: RUNNING  READY
Preemptive
Also when BLOCKED READY transition
Also on timer (forced call to yield upon intr exit)
RUNNING
READY
BLOCKED
Process
must wait
for event
Event arrived
Scheduler
picks process
preempted
CS 5204 Fall 2015

16. CPU Scheduling Goals Minimize latency Maximize throughput Fairness
Can mean (avg) completion time
Can mean (avg) response time
Maximize throughput
Throughput: number of finished jobs per time-unit
Implies minimizing overhead (for context-switching, for scheduling algorithm itself)
Requires efficient use of non-CPU resources
Fairness
Minimize variance in waiting time/completion time
CS 5204 Fall 2015

17. Scheduling Constraints
Reaching those goals is difficult, because
Goals are conflicting:
Latency vs. throughput
Fairness vs. low overhead
Scheduler must operate with incomplete knowledge
Execution time may not be known
I/O device use may not be known
Scheduler must make decision fast
Approximate best solution from huge solution space
CS 5204 Fall 2015

18. First Come First Serve
Schedule processes in the order in which they arrive
Run until completion (or until they block)
Simple!
Example:
Q.: what is the average
completion time? 2 7 20 22 27
CS 5204 Fall 2015

19. FCFS (cont’d) Disadvantage: completion time depends on arrival order
Unfair to short jobs
Possible Convoy Effect:
1 CPU bound (long CPU bursts, infrequent I/O bursts), multiple I/O bound jobs (frequent I/O bursts, short CPU bursts).
CPU bound process monopolizes CPU: I/O devices are idle
New I/O requests by I/O bound jobs are only issued when CPU bound job blocks – CPU bound job “leads” convoy of I/O bound processes
FCFS not usually used for CPU scheduling, but often used for other resources (network device)
CS 5204 Fall 2015

20. Round-Robin
Run process for a timeslice (quantum), then move on to next process, repeat
Decreases avg completion if jobs are of different lengths
No more unfairness to short jobs!
Q.: what is the average
completion time? 5 8 27
CS 5204 Fall 2015

21. Round Robin (2) What if there are no “short” jobs?7 14 21
Q.: what is the average completion time?
What would it be under FCFS?
CS 5204 Fall 2015

22. Round Robin – Cost of Time Slicing
Context switching incurs a cost
Direct cost (execute scheduler & context switch) + indirect cost (cache & TLB misses)
Long time slices  lower overhead, but approaches FCFS if processes finish before timeslice expires
Short time slices  lots of context switches, high overhead
Typical cost: context switch < 10µs
Time slice typically around 100ms
Note: time slice length != interval between timer interrupts where periodic timers are used
CS 5204 Fall 2015

23. Shortest Process Next (SPN)
Idea: remove unfairness towards short processes by always picking the shortest job
If done nonpreemptively also known as:
Shortest Job First (SJF), Shortest Time to Completion First (STCF)
If done preemptively known as:
Shortest Remaining Time (SRT), Shortest Remaining Time to Completion First (SRTCF)
CS 5204 Fall 2015

24. SPN (cont’d) Provably optimal with respect to avg waiting time:
Moving shorter job up reduces its waiting time more than it delays waiting time of longer job that follows
Advantage: Good I/O utilization
Disadvantage:
Can starve long jobs 2 7 27
Big Q: How do we know the length of a job?
CS 5204 Fall 2015

25. Practical SPN Usually don’t know (remaining) execution time
Exception: profiled code in real-time system; or worst-case execution time analysis (WCET)
Idea: determine future from past:
Assume next CPU burst will be as long as previous CPU burst
Or: weigh history using (potentially exponential) average: more recent burst lengths more predictive than past CPU bursts
Note: for some resources, we know or can compute length of next “job”:
Example: disk scheduling (shortest-seek time first)
CS 5204 Fall 2015

26. Aside
Computation Takes Time, But How Much?
Reinhard Wilhelm, Daniel Grund  Communications of the ACM, Vol. 57 No. 2, Pages /
CS 5204 Fall 2015

27. Multi-Level Feedback Queue Scheduling
Kleinrock 1969
Want:
preference for short jobs (tends to lead to good I/O utilization)
longer timeslices for CPU bound jobs (reduces context-switching overhead)
Problem:
Don’t know type of each process – algorithm needs to figure out
Use multiple queues
queue determines priority
usually combined with static priorities (nice values)
many variations of this idea exist
CS 5204 Fall 2015

28. MLFQS
Processes start in highest queue
MIN
MAX
Higher Priority 1 2 4 3
Longer Timeslices
Process that
use up their
time slice move
down
Processes that starve move up
Higher priority queues are served before lower-priority ones - within highest-priority queue, round-robin
Only ready processes are in this queue - blocked processes leave queue and reenter same queue on unblock
CS 5204 Fall 2015

29. Basic Scheduling: Summary
FCFS: simple
unfair to short jobs & poor I/O performance (convoy effect)
RR: helps short jobs
loses when jobs are equal length
SPN: optimal average waiting time
which, if ignoring blocking time, leads to optimal average completion time
unfair to long jobs
requires knowing (or guessing) the future
MLFQS: approximates SPN without knowing execution time
Can still be unfair to long jobs
CS 5204 Fall 2015

30. CPU Scheduling
Part II

31. Case Study: 2.6 Linux Scheduler (pre 2.6.23)
nice=19
140
Variant of MLFQS
140 priorities
0-99 “realtime”
nonrealtime
Dynamic priority computed from static priority (nice) plus “interactivity bonus”
Processes
scheduled
based on dynamic
priority
SCHED_OTHER
nice=0
120
nice=-20
100
“Realtime”
processes
scheduled based on
static priority
SCHED_FIFO
SCHED_RR
CS 5204 Fall 2015

32. Linux Scheduler (2)
Instead of recomputation loop, recompute priority at end of each timeslice
dyn_prio = nice + interactivity bonus (-5…5)
Interactivity bonus depends on sleep_avg
measures time a process was blocked
2 priority arrays (“active” & “expired”) in each runqueue (Linux calls ready queues “runqueue”)
CS 5204 Fall 2015

33. Linux Scheduler (3) Finds highest-priority ready thread quickly
struct prio_array {
unsigned int nr_active;
unsigned long bitmap[BITMAP_SIZE];
struct list_head queue[MAX_PRIO]; };
typedef struct prio_array prio_array_t;
/* find the highest-priority ready thread */
idx = sched_find_first_bit(array->bitmap);
queue = array->queue + idx;
next = list_entry(queue->next, task_t, run_list);
/* Per CPU runqueue */
struct runqueue {
prio_array_t *active;
prio_array_t *expired;
prio_array_t arrays[2]; … }
Finds highest-priority ready thread quickly
Switching active & expired arrays at end of epoch is simple pointer swap (“O(1)” claim)
CS 5204 Fall 2015

34. Linux Timeslice Computation
Linux scales static priority to timeslice
Nice [ … … 19 ] maps to [800ms … 100 ms … 5ms]
Various tweaks:
“interactive processes” are reinserted into active array even after timeslice expires
Unless processes in expired array are starving
processes with long timeslices are round-robin’d with other of equal priority at sub-timeslice granularity
CS 5204 Fall 2015

35. History
Variants of MLFQS dominant until a few years ago; still used in Windows kernel
Accompanied by belief that online scheduler must be O(1) with small c
MLFQS are easily manipulated and do not guarantee fair (“proportional”) CPU assignments
Another problem is accuracy of accounting – sampling charges entire tick to process that happened to be running at that point
CS 5204 Fall 2015

36. Accuracy of accounting
Instead of relying on sampling, modern versions of OS use cycle counters or high-precision timers to accurate determine a process’s recent CPU usage.
This also makes it easy to exclude time spent in IRQ handling that should not be charged to a process
See [Inside the Vista Kernel] for a graph
CS 5204 Fall 2015

37. Linux’s CFS
Linux went a step further and reinvented WFQ (of course without any credit), implemented in its “CFS”
“completely fair scheduler”
O (log n) – red/black tree
But does not aim to support really large n
But, as we’ll see, WFQ does not automatically give precedence to I/O bound apps; required a lot of “Interactivity improvements” to tune it heuristically
Well-known trade-off between fairness & latency
CS 5204 Fall 2015

38. Proportional Share Scheduling
Aka “Fair-Share” Scheduling
None of algorithms discussed so far provide a direct way of assigning CPU shares
E.g., give 30% of CPU to process A, 70% to process B
Proportional Share algorithms do by assigning “tickets” or “shares” to processes
Process get to use resource in proportion of their shares to total number of shares
Lottery Scheduling, Weighted Fair Queuing/Stride Scheduling [Waldspurger 1995]
CS 5204 Fall 2015

39. Lottery Scheduling Idea: number tickets between 1…N
every process gets pi tickets according to importance
process 1 gets tickets [1… p1-1]
process 2 gets tickets [p1… p1+p2-1] and so on.
Scheduling decision:
Hold a lottery and draw ticket, holder gets to run for next time slice
Nondeterministic algorithm
Q.: how to implement priority donation?
CS 5204 Fall 2015

40. Weighted Fair Queuing Uses ‘per process’ virtual time
Increments process’s virtual time by a “stride” after each quantum, which is defined as (process_share)-1
Choose process with lowest virtual finishing time
‘virtual finishing time’ is virtual time + stride
Also known as stride scheduling
Linux now implements a variant of WFQ/Stride Scheduling as its “CFS” completely fair scheduler
CS 5204 Fall 2015

41. WFQ Example (A=3, B=2, C=1)
Ready Queue is sorted by Virtual Finish Time (Virtual Time at end of quantum if a process were scheduled)
Time
Task A
Task B
Task C
Ready Queue
Who Runs
One scheduling epoch. A ran 3 out of 6 quanta, B 2 out of 6, C 1 out of 6.
This process will repeat, yielding proportional fairness.
1/3
1/2 1
A (1/3) B (1/2) C (1) A
2/3
B (1/2) A (2/3) C (1) B 2
A (2/3) C(1) B(1) 3
C(1) B(1) A(1)  C 4
B(1) A(1) C(2) 5
3/2
A(1) B(3/2) C(2) A  6
4/3
A (4/3) B(3/2) C(2)
CS 5204 Fall 2015

42. WFQ (cont’d) WFQ requires a sorted ready queue
Linux now uses R/B tree
Higher complexity than O(1) linked lists, but appears manageable for real-world ready queue sizes
Unblocked processes that reenter the ready queue are assigned a virtual time reflecting the value that their virtual time counter would have if they’d received CPU time proportionally
Accommodating I/O bound processes still requires fudging
In strict WFQ, only way to improve latency is to set number of shares high – but this is disastrous if process is not truly I/O bound
Linux uses “sleeper fairness,” to identify when to boost virtual time; similar to the sleep average in old scheduler
CS 5204 Fall 2015

43. Linux SMP Load Balancing
static void double_rq_lock( runqueue_t *rq1, runqueue_t *rq2) {
if (rq1 == rq2) {
spin_lock(&rq1->lock);
} else {
if (rq1 < rq2) {
spin_lock(&rq2->lock); }
Runqueue is per CPU
Periodically, lengths of runqueues on different CPU is compared
Processes are migrated to balance load
Aside: Migrating requires locks on both runqueues
CS 5204 Fall 2015

44. Real-time Scheduling
Real-time systems must observe not only execution time, but a deadline as well
Jobs must finish by deadline
But turn-around time is usually less important
Common scenario are recurring jobs
E.g., need 3 ms every 10 ms (here, 10ms is the recurrence period T, 3 ms is the cost C)
Possible strategies
RMA (Rate Monotonic)
Map periods to priorities, fixed, static
EDF (Earliest Deadline First)
Always run what’s due next, dynamic
CS 5204 Fall 2015

45. EDF – Example Assume deadline equals period (T). A B C Hyper-period
Task T C A 4 1 B 8 12 3
Assume deadline equals period (T). A B C 5 10 15 20 25
Hyper-period
CS 5204 Fall 2015

46. EDF – Example Assume deadline equals period (T). A B C Task T C A 4 18 12 3
Assume deadline equals period (T). A B C 5 10 15 20 25
CS 5204 Fall 2015

47. EDF – Example Assume deadline equals period (T). A B C Task T C A 4 18 12 3
Assume deadline equals period (T). A B C 5 10 15 20 25
CS 5204 Fall 2015

48. EDF – Example Assume deadline equals period (T). A B C Task T C A 4 18 12 3
Assume deadline equals period (T).
Lexical order tie breaker (C > B > A) A B C 5 10 15 20 25
CS 5204 Fall 2015

49. EDF – Example Assume deadline equals period (T). A B C Task T C A 4 18 12 3
Assume deadline equals period (T). A B C 5 10 15 20 25
CS 5204 Fall 2015

50. EDF – Example Assume deadline equals period (T). A B C Task T C A 4 18 12 3
Assume deadline equals period (T). A B C 5 10 15 20 25
CS 5204 Fall 2015

51. EDF – Example Assume deadline equals period (T). A B C Task T C A 4 18 12 3
Assume deadline equals period (T). A B C 5 10 15 20 25
CS 5204 Fall 2015

52. EDF – Example Assume deadline equals period (T). A B C Task T C A 4 18 12 3
Assume deadline equals period (T). A B C 5 10 15 20 25
CS 5204 Fall 2015

53. EDF – Example Assume deadline equals period (T). A B C Pattern repeats
Task T C A 4 1 B 8 12 3
Assume deadline equals period (T). A B C 5 10 15 20 25
Pattern repeats
CS 5204 Fall 2015

54. EDF Properties Feasibility test: U = 100% in example Bound theoretical
Sufficient and necessary
Optimal
CS 5204 Fall 2015

55. Scheduling Summary OS must schedule all resources in a system
CPU, Disk, Network, etc.
CPU Scheduling affects indirectly scheduling of other devices
Goals for general purpose schedulers:
Minimizing latency (avg. completion or waiting time)
Maximing throughput
Provide fairness
In Practice: some theory, lots of tweaking
CS 5204 Fall 2015