April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 1 Process Management -- Scheduling ECEN5043 Software Engineering of Multi-Program Systems University of Colorado, Boulder

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 2 Sources Content taken from Tanenbaum text, section 2.5, on Scheduling Does not include material on scheduling real time systems with hard deadlines -- covered later

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 3 Scheduling In multi-programming –often multiple processes compete for the CPU at the same time –in other words, the CPU is a shared resource Scheduler decides which of two competing processes to run Process scheduling first, then thread scheduling

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 4 Historical Introduction Batch – card images spooled onto magnetic tape –Scheduling algorithm: run the next one on the tape Timesharing – multiple users waiting for service –more complicated algorithm Depending on the scheduling algorithm, it’s possible to greatly impact –perceived performance –user satisfaction

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 5 Enter personal computers What’s different about personal computers? –Only one active process usually –CPUs so fast now – it is rarely a scarce resource Now limited by the rate user can present input, not by the rate the CPU can process it If two active processes, same user is probably waiting for both High-end networked workstations and servers... their usage characteristics are different than PCs 

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 6 High-end Networked Workstations & Servers Multiple processes compete Scheduling is important in this context Choices can greatly impact perceived performance

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 7 Too many Process Switches per Second Eat Up CPU Time Switched From Switch from user mode to kernel mode Store registers in a process control block table to be reloaded later Save memory map Invalidate memory cache Switched To Load MMU with the memory map of the new process Start the new process

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 8 Process Behavior CPU cycles – CPU is in use –Most of time computing – compute bound I/O cycles –Process enters the blocked state, waiting for an external device to complete its work –Short CPU bursts and frequent I/O waits Key factor – length of the CPU burst I/O bound – does not compute much in between I/O accesses; not because I/O access takes a long time What is impact of increasing CPU speeds? Should I/O bound process get run-time priority?

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 9 When Is Scheduling Needed? When a new process is created –Run the parent –Run the child –Both processes are in ready state so it can go either way When a process exits –can no longer run (no longer exists) –choose another from the ready queue –if ready queue empty, system runs an idle process usually

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 10 When Is Scheduling Needed? (continued) When a process is blocked and waiting (on an IO device, a semaphore) When an I/O interrupt occurs, scheduler has choices –If source of interrupt was a newly finished I/O device, the process that was blocked while waiting for this device is a likely candidate to run –Perhaps the process that was running at the time of the interrupt should go on running –Perhaps some third process should run

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 11 Special case of clock interrupts Sometimes the hardware clock provides periodic interrupts –A scheduling decision can be made at each clock interrupt or every k th one

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 12 Non-preemptive scheduling algorithm Picks a process & lets it run until it –blocks or –voluntarily releases the CPU (suspends itself) After clock interrupt processing, the previously running process resumes

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 13 Preemptive scheduling algorithm Picks a process & lets it run for a max of some fixed time If still running, it is suspended Scheduler picks another process, if any REQUIRES a clock interrupt at the end of the time interval so that the scheduler regains control of the CPU If no clock... no preemptive scheduling.

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 14 Types of Scheduling Algorithms What the scheduler should optimize for is not the same in all systems and environments Batch Interactive Real Time

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 15 Batch Scheduling Needs No impatient users Non-preemptive algorithms are ok Preemptive algorithms with long processing periods Advantages?

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 16 Interactive Environments Scheduling Needs Preemption essential to keep one process from hogging the CPU “Hogging” may not be intentional but could happen because of a bug

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 17 Real Time Constraints Preemption may NOT be needed; may defeat goals Processes “know” they can’t run for long periods of time – they do their work and block (suspend themselves) How different from interactive systems? –RT programs further the real time application –Interactive systems are general purpose and run arbitrary programs Programs are probably not cooperative, may be malicious What is impact on design of multi-program systems in an interactive environment where these programs are intended to be cooperative??

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 18 What’s a Good Scheduling Algorithm? Environment dependent (batch, interactive, RTS) Others are “always true” –Fairness –Policy enforcement –Balance Or are they? –Fairness  comparable processes get comparable service –Policy enforcement  policies vary with equivalence classes of processes –Balance  all parts of the system are busy when possible but not soooo busy as to hurt throughput.

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 19 BATCH goals Can incorrectly influence managers of interactive systems because they used to be good goals... Throughput Turnaround time CPU utilization Conflicts in these goals?

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 20 Interactive Systems – scheduling goals Goals for timesharing systems are not the same as those for servers Response time –If there are background processes, giving priority to interactive requests is perceived as good service. Proportionality – relative expectations of response time based on perceived complexity Scheduler cannot help response time but can influence appearance by process order selection

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 21 Real Time Systems – scheduling goals Characteristics: –Deadlines that must be met –Deadlines that usually should be met Foremost goal – meet deadlines Predictability (related to jitter) –Example: sound quality –Example: cruise control acceleration –Example: video “smoothness” Coming events: real time scheduling emphasis two weeks from now (guest speaker next week)

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 22 Batch-only Scheduling choices Why study these? –To avoid using these in interactive multiprogramming systems –To use them in special situations where best suited First-Come First-Served Shortest Job First Shortest Remaining Time Next Three Level Scheduling

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 23 First-Come First-Served What is it? –Processes assigned CPU in the order they requested it –Single ready queue –When a running process blocks, next in queue is run –When a blocked process becomes ready, it is put at the end of the queue Strength – easy to understand and program Fair – like waiting in line for Super Bowl tickets is fair Powerful disadvantage?

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 24 Shortest Job First Non-preemptive In certain environments, easy to estimate how long certain processes will take on average When several equally important tasks are in the input queue, scheduler picks the shortest job first. Suppose jobs A, B, C, D take 8, 4, 4, 4 minutes respectively and arrive in that order –How long will each take with FCFS? –How long will each take with SJF? If all jobs available simultaneously, SJF is provably optimal

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 25 Shortest Remaining Time Runs Next Preemptive version of SJF Scheduler always chooses the process whose remaining run time is the shortest. (Only useful in an environment where run times can be estimated or known.) When new process arrives, its total time is compared to current process’ remaining time If new job needs less time to finish, current process is suspended Result: new short jobs get good service.

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 26 Three Level Scheduling Batch systems allow scheduling at 3 levels

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 27 Interactive Systems – Scheduling All of these can also be used as the CPU scheduler in batch systems 3-level scheduling not possible but 2-level is (memory and CPU) Round-robin Priority Multiple Queues Shortest process next Guaranteed scheduling Lottery scheduling Fair-share scheduling

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 28 Round-robin -- interactive One of the oldest, simplest, fairest, most widely used Each assigned a time interval called a quantum If still running at end of interval, preempted and CPU given to another process If process blocked or finished before quantum elapsed, CPU switches at that time When a process uses up its quantum, it is put at the end of the list – assumption: all are equally important Length of the quantum? –context switching takes time

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 29 Scheduling in Interactive Systems (1) Round Robin Scheduling –list of runnable processes –list of runnable processes after B uses up its quantum

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 30 Priority -- interactive Priority scheduling takes external factors of importance into account The runnable process with the highest priority runs next Even PC environment has processes with different priorities How avoid high-priority process running forever? –scheduler may decrease its priority with each clock tick; when priority drops below next highest, switch –OR: each process may have max quantum; when used up, choice of next process based on priority

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 31 Priority allocation Priorities can be assigned statically or dynamically Static assignment –may be associated with user id –may be based on category Dynamic assignment –to achieve system goals –recognize characteristic and assignment priority e.g. Good service to I/O bound processes: set the priority to 1/f where f is the fraction of the last quantum the process used

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 32 Scheduling in Interactive Systems (2) A scheduling algorithm with four priority classes. Priority scheduling among classes Round-robin scheduling within each class. When will a process of Priority 3 get to run? May have to occasionally adjust priorities to avoid starvation of low priority tasks.

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 33 Multiple Queues -- interactive Priority classes Processes in highest class run for one quantum. Processes in next highest class run for two quanta. Processes in next highest class run for four quanta. etc. If a process uses up all the quanta allocated before blocking, move it down one class. This was the solution used in CTSS where process switching was very slow because the 7094 could only hold one process in memory. All switches were to/from disk.

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 34 First Impressions If a process needed to run for a long time at first and then became interactive: –when a carriage return (enter) was typed at a terminal, the process belonging to that terminal was moved to the next highest priority class –inherent problem -- what is it?

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 35 Dynamic adaptation To favor interactive users and processes over background processes –Assign, say, 4 priority classes (one example used “terminal”, “I/O”, “short”, “long”. –A process waiting for terminal input is awakened -- goes into highest priority –A process waiting for disk block goes into 2nd category when it becomes ready –If process still running when its quantum expires, goes into 3rd category –If process uses up its quantum too many times in a row without blocking, moves down to last queue

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 36 Shortest process next -- interactive Because shortest job first always produces the minimum average response time for batch systems, it was tried on interactive systems. Interactive processes follow the pattern of wait for command, execute command, (repeat) If each execution of a command is a separate “job”, then we could run the shortest one first. Which of the currently runnable processes is the shortest one?

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 37 Guaranteed scheduling -- interactive Different approach -- make real promises about performance to users and then live up to them Example: If there are n users logged in while you are working, you will receive about 1/n of the CPU power. To make good, –system keeps track of how much CPU each process has had since its creation. –Computes the amount of CPU each is entitled to (time since creation divided by n) –Compute the ratio of actual CPU time consumed to CPU time entitled –Run the process with the lowest ratio until its ratio passes its closest competitor

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 38 Lottery scheduling -- interactive Sometimes difficult to implement a way to live up to promises Lottery scheduling gives similarly predictable results with a simpler implementation Give processes “lottery tickets” for system resources such as CPU time When scheduling decision needs to be made, lottery ticket chosen at random and the process holding that ticket gets the resource. Applied to CPU scheduling, system might hold a lottery 50 times a second with each winner getting 20ms of CPU time as its prize. Ways to manage this?

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 39 Interesting properties of Lottery Scheduling Highly responsive -- if new process shows up and gets some tickets, it has a chance of winning right away Cooperating processes may exchange tickets –When client process sends a message to a server process and then blocks, it can give all its tickets to the server... –When server is done, it returns the tickets... –And in absence of clients, server needs no tickets Solves some hard problems

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 40 Fair-share scheduling – interactive Have been assuming each process is scheduled on its own, independent of knowing its owner Some systems schedule processes based on usage allocated to each user, independent of the number of processes that user has –2 users each promised 50% of the CPU; they each get 50% no matter how many processes they have in existence.

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 41 Policy versus Mechanism Separate what is allowed to be done with how it is done –A process knows which of its children threads are important and need priority –None of the algorithms discussed so far accept input from user processes about scheduling decisions Separate scheduling mechanism from scheduling policy Scheduling algorithm parameterized somehow –mechanism in the kernel (operating system) Parameters filled in by user processes –policy set by user process

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 42 Thread Scheduling - 1 When several processes have multiple threads, we have 2 levels of parallelism: processes and threads Scheduling depends on whether user-level threads or kernel-level threads are supported User-level threads –kernel is not aware of them; it schedules processes –no clock interrupt to multiprogram threads –“A1” may run as long as it wants to –If it uses up all of the process’ quantum, kernel picks another process –When A resumes, A1 resumes –A1’s antisocial behavior is locally limited

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 43 User-Level Thread Scheduling -- 2 Suppose A’s threads have 5 ms each within a 50 ms quantum Each runs for a little while, yields the CPU back to the thread scheduler See figure on next slide

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 44 User-Level Thread Scheduling -- 3 Possible scheduling of user-level threads 50-msec process quantum threads run 5 msec/CPU burst

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 45 User-level Thread Scheduling -- 4 Algorithm used by the run-time system can be any we’ve looked at Round-robin and priority scheduling are most common Only constraint: there is no clock to interrupt a thread that has run over long

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 46 Kernel-Level Thread Scheduling Kernel picks a particular thread to run May or may not take into account the process owner Thread can be given a quantum; suspended if it exceeds Example: see next slide

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 47 Thread Scheduling -- 4 Possible scheduling of kernel-level threads 50-msec process quantum threads run 5 msec/CPU burst

April 1, 2002 ECEN Software Engineering of Multi-Program Systems -- University of Colorado -- Scheduling 48 User-level vs. kernel-level threads Thread switch takes a small number of machine instructions Thread block suspends entire process Can use an application-specific thread scheduler to tune an application Thread switch requires a full context switch Having a thread block does not suspend the entire process Kernel knows it’s more expensive to switch to thread in different process; can use this info