Processes and Schedulers

What is a Process Process: An execution stream and its associated state Execution Stream – Set of instructions – “Thread of control” Process State – Hardware state Privilege level, segments, page tables – OS State Priority, I/O Buffers, Heap, memory map – Resource state I/O requests An abstraction to make it easier to program both OS and applications – Encapsulate state into manageable unit

Programs and Processes A process is not a program Program: Static code and static data int foo() { return 0; } int main() { foo(); return 0; } Program int foo() { return 0; } int main() { foo(); return 0; } Heap Stack Registers Process OS can host multiple processes of same program – E.g. many users can run ‘ls’ at the same time Once program can invoke multiple processes – E.g. make runs many processes to compile code No one-to-one mapping between programs and processes

Threads and Processes A process is different than a thread – Conceptually (On Linux it is more complicated) Thread: Separate execution streams in same address space – “Lightweight process” int foo() { return 0; } int main() { foo(); return 0; } Heap Stack Registers Process Can have multiple threads within a process int foo() { return 0; } int main() { foo(); return 0; } Heap Threads Stack Registers Stack Registers

System Classification Uniprogramming: Only one process at a time – Examples: Original systems and older PC Oses DOS – Advantages: Easier for OS designer – Disadvantages: Terrible utilization, poor usability Multiprogramming: Multiple processes at a time – Examples: Every modern OS you use – Note: Multiprogramming is different from multiprocessing Multiprocessing: Systems with multiple processors – Advantages: Better utilization and usability – Disadvantages: Complex OS design

Multiprogramming OS requirements for multiprogramming – Policy to determine process to run – Mechanism to switch between processes – Methods to protect processes from one another Memory management system Separation of policy and mechanism – Recurring theme in OS design – Policy: Decision maker based on some metric Scheduler – Mechanism: Low level code that implements the decision Dispatcher/Context Switch

Multiprogramming and Memory Many OSes didn’t do a very good job of combining these Early PC OSes didn’t protect memory – MacOS and Windows – Each process could access all of memory Same address space Basically a giant multithreaded environment All modern OSes not include memory map in PCB – Processes cannot access each other memory

Dispatch Mechanism OS maintains list of all processes Each process has a mode – Running: Executing on the CPU – Ready: Waiting to execute on CPU – Blocked: Waiting for I/O or synchronization with another thread Dispatch Loop while (1) { run process for a while; stop process and save its state; load state of another process; } How does dispatcher gain control? What execution state must be saved/restored?

How does dispatcher gain control? Must change from user to system mode – Problem: Only one CPU, and CPU can only do one thing at a time – A user process running means the dispatcher isn’t Two ways OS gains control Traps: Events caused by process execution – System calls, page faults, Exceptions (segfault, etc) Hardware interrupts: Events external to process – Typing at keyboard, network packet arrivals – Control switch to OS via Interrupt Service Routine (ISR) How does OS guarantee it will regain control?

Approaches to dispatcher Option 1: Cooperative multitasking – Trust process to invoke dispatcher – Linux: Default for kernel code schedule() – Disadvantage: A mistake in one part of the code can lock up entire system Option 2: True multitasking – Configure hardware to periodically invoke dispatcher – Hardware generated timer interrupt Timer ISR invokes dispatcher – Linux: Enabled for user processes HZ – Processes run for some multiple of timer “ticks” (interrupts) Process time slice

What state must be saved? OS must track state of processes – On every trap/interrupt save process state in “process control block” (PCB) Why on every trap/interrupt? Data structure problem: How to manages all PCBs Information stored in PCB – Execution state General registers, control registers, CPU flags, RSP, RIP, page tables – OS state Memory map, heap space – I/O status Open files and sockets – Scheduling information Execution mode, priority – Accounting information Owner, PID – Plus lots more

Context Switch implementation Machine dependent code (Assembly!) – Different for MIPS, ARM, x86, etc. – Save process state to PCB Tricky: OS must save state without changing state Requires special hardware support – Save process state on each trap/interrupt – Very nasty x86 has TSS (PCB) that most OSes avoid

Process Creation Two ways to create a process – Build one from scratch – Clone an existing one Option 1: From scratch (Windows – CreateProcess(…)) – Load specified code and data into memory – Create empty call stack – Create and initialize PCB (make it look like a context switch) – Add process to ready list Option 2: Cloning (UNIX – fork()) – Stop current process and save its state – Copy code, data, stack and PCB – Add new Process PCB to ready list – Do we really need to copy everything?

Creating a process (Windows and UNIX) BOOL WINAPI CreateProcess( _In_opt_ LPCTSTR lpApplicationName, _Inout_opt_ LPTSTR lpCommandLine, _In_opt_ LPSECURITY_ATTRIBUTES lpProcessAttributes, _In_opt_ LPSECURITY_ATTRIBUTES lpThreadAttributes, _In_ BOOL bInheritHandles, _In_ DWORD dwCreationFlags, _In_opt_ LPVOID lpEnvironment, _In_opt_ LPCTSTR lpCurrentDirectory, _In_ LPSTARTUPINFO lpStartupInfo, _Out_ LPPROCESS_INFORMATION lpProcessInformation ); int fork(); Windows UNIX

Creating processes in UNIX Combination of fork() and exec(…) – fork(): Clone current process – exec(…): copy new program on top of current process int main() { int pid; char * cmd = “/bin/sh”; pid = fork(); if (pid == 0) { // child process exec(cmd); // exec does not return, WE NEVER GET HERE } else { // parent process – wait for child to finish wait(pid); } Advantage: Flexible, clean, simple

Process Abstraction Processes are a low level component – Provide an abstraction to build on Fundamental OS design – Provide abstract units (resources) that high level policies can act on Resources – Resources are high level units managed by OS – CPU time, memory, disk space, I/O bandwidth How does the OS manage resources?

Resources Preemptible – Resource can be taken away and used by somebody else – Example: CPU Non-preemptible – One a resource is assigned it can only be returned voluntarily – Example: Disk space OS must balance set of resources and requests for those resources – OS management depends on type of resource

Decisions about Resources Allocation: Which process gets which resource – Which resources should each process get? – Space sharing: Control concurrent access to resource – Implication: Resources are not easily preemptible – Example: disk space Scheduling: How long process keeps resource – In which order should requests be serviced – Time sharing: More resources requested than exist – Implication: Resource is preemtible – Example: CPU time

Role of Dispatcher vs. Scheduler Dispatcher – Low-level mechanism – Responsibility: Context-switch Change mode of old process to either WAITING or BLOCKED Save execution state of old process in PCB Load state of new process from PCB Change mode of new processes to RUNNING Switch to user mode privilege Jump to process instruction Scheduler – Higher level policy – Responsibility: Decide which process to dispatch to CPU could be allocated – Parallel and Distributed Systems

Scheduling Performance Metrics Minimize response time – Increase interactivity (responsiveness of user interfaces) Maximize resource utilization – Keep CPU and disks busy Minimize overhead – Reduce context switches (number and cost) Distributed resources fairly – Give each user/process same percentage of CPU

Scheduling Algorithms Process (job) model – Process alternates between CPU and I/O bursts – CPU bound job: long CPU bursts – I/O bound job: short CPU bursts Don’t know before execution – Need to handle full range of possible workloads Scheduling Algorithms – First-Come-First-Served (FCFS) – Shortest Job First (SJF) or Shortest-Time-Completion-First (STCF) – Round-Robin (RR) – Priority Scheduling – Other scheduling algorithms that are actually used

First Come First Served (FCFS) Simplest Scheduling algorithm – First job that requests CPU is allocated CPU – Nonpreemptive Advantage: Simple Implementation with FIFO queue Disadvantage: Response time depends on arrival order – Unfair to later jobs (especially if the system has long jobs) Job A Job B Job C Time CPU Uniprogramming: Run job to completion Multiprogramming: Put job at back of queue when performing I/O

Convoy Effect Short running jobs stuck waiting for long jobs – Example: 1 CPU bound job, 3 I/O bound jobs Problems – Reduces utilization of I/O devices – Hurts response time of short jobs A A B B C C Time CPU C C A A B B C C C C A A B B C C C C A A B B C C C C Idle Disk

Shortest Job First Minimizes average response time Job A Job B Job C Time CPU FCFS if simultaneous arrival Provably optimal (given no preemption) to reduce response time – Short job improved more than long job is hurt Not practical: Cannot know burst lengths (I/O + CPU) – Can only use past behavior to predict future behavior

Shortest Time to Completion First (STCF) SJF with preemption – New process arrives w/ short CPU burst – Shorter than remaining time of current job A A B B D D Time CPU A A C C B B D D Time CPU A A C C SJF without preemption Job Submission: A at time 0, B/C/D at time t t0

Shortest Remaining Processing Time (SRPT) STCF for batch workloads Used in distributed systems – Provides maximum throughput (transactions/sec) – Minor risk of starvation Very popular in web servers and similar systems

Round Robin (RR) Practical approach to support time-sharing – Run job for a time-slice and then go to back of queue – Preempted if still running at end of time-slice Advantages – Fair allocation of CPU across jobs – Low average response time when job lengths vary widely Avoids worst case scenarios and starvation A A A A CPU B B C C A A B B C C A A B B Time

Disadvantages of Round Robin Poor average response time when job sizes are identical – E.g. 10 jobs each require 10 time slices – All complete after 100 time slices – Even FCFS is better How large should the time slice be? – Depends on the workload! – Tradeoff between throughput and responsiveness Batch vs. interactive workloads

Priority based scheduling Priorities assigned to each process – Run highest priority job in system (that is ready) – Round robin among equal priority levels Aside: How to parse priority numbers – Is low high or is high high? (Depends on system) Static vs. Dynamic priorities – Some jobs have static priority assignments (kernel threads) – Others need to be dynamic (user applications) – Should priorities change as a result of scheduling decisions?

Real world scheduling Current schedulers exhibit combinations of above approaches – Include priorities, round robin queues, queue reordering, and much more – There is no single good algorithm – Nor is there a way to even measure “goodness” Most schedulers designed based on heuristics and best guesses of potential workloads – Linux has a lot of complexity to detect batch vs. interactive processes (can change during execution) Many times a scheduler will work great 99% of the time but completely fall over for 1% of workloads