0. Unit OS4: Scheduling and Dispatch
4.1. The Concept of Processes and Threads
Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze

1. Copyright Notice © 2000-2005 David A. Solomon and Mark Russinovich
These materials are part of the Windows Operating System Internals Curriculum Development Kit, developed by David A. Solomon and Mark E. Russinovich with Andreas Polze
Microsoft has licensed these materials from David Solomon Expert Seminars, Inc. for distribution to academic organizations solely for use in academic environments (and not for commercial use)

2. Roadmap for Section 4.1. The Process Concept Thread States
Context Switches
Approaches to CPU Scheduling
Multithreading Models
A process can be thought of as a program in execution. To accomplish its task, a process will need certain resources – such as CPU time, memory, files, and I/O devices. These resources are allocated to the process by the operating system, either when the process is created or while it is executing.
Windows, as many modern operating systems, provides features for a process to contain multiple threads of control. A thread is the basic unit of CPU utilization. It comprises a thread identifier, a program counter, a register set and a stack. It shares with other threads in the same process its code section, data section, and resources, such as file handles, ports, and network endpoints.
The operating system is responsible for creation and deletion of user and system processes and threads, for the scheduling of threads, and the provisioning of mechanisms for concurrency control and communication.
This section explains the notions of a process and a thread and gives an overview over scheduling criteria and approaches.

3. Process Concept An operating system executes programs:
Batch system – jobs
Time-shared systems – user programs or tasks
Process – a program in execution
Process execution must progress sequentially
A process includes:
CPU state (one or multiple threads)
Text & data section
Resources such as open files, handles, sockets
Traditionally, processes used to be units of scheduling (i.e. no threads)
However, like most modern operating systems, Windows schedules threads
Our discussion assumes thread scheduling
The fundamental task of any modern operating system is process management. The OS must allocate resources to processes, allow processes to share and exchange information, and protect resources and processes from other processes. To meet these requirements, the OS must maintain data describing state and resource ownership for each process.
In most modern operating systems, the introduction of threads extends the notion of processes. In a multithreaded system, the process retains the attributes of resource ownership, while the attribute of multiple, concurrent execution streams is a property of threads running within a process.
In Windows, the thread is the fundamental unit of scheduling.

4. Thread States Five-state diagram for thread scheduling:
init: The thread is being created
ready: The thread is waiting to be assigned to a CPU
running: The thread’s instructions are being executed
waiting: The thread is waiting for some event to occur
terminated: The thread has finished execution
interrupt quantum expired
init
terminated
Newly created threads start there live in “init” state. After appropriate data structures are initialized (thread control block - TCB), the thread’s state is changed to “ready”. From “ready” state, a thread may get selected for execution and its state is changed by the scheduler to “running”. For a given system, there is exactly one thread in “running” state per processing unit (note that a CPU might have multiple processing units - multicore/hyperthreading).
When the “running” thread issues an I/O request, it might block and change its state into “waiting”. Alternatively, a “running” thread’s time slice (quantum) may expire, in which case the thread’s state is changed back to “ready”. A “waiting” thread can move into “ready” state once the reason for its blocking disappears (I.e.; the I/O operation completes). Finally, a “running” thread might execute its last instruction - in which case its state changes to “terminated”.
Operating systems often use heuristics to determine which one of the threads in “ready” state to select for execution. The Windows OS uses a priority-based scheme and maintains 32 ready queues of different priorities.
admitted
scheduler dispatch
exit
ready
running
waiting
I/O or event completion
waiting for
I/O or event

5. Process and Thread Control Blocks
Information associated with each process: Process Control Block (PCB)
Memory management information
Accounting information
Process-global vs. thread-specific
Information associated with each thread: Thread Control Block (TCB)
Program counter
CPU registers
CPU scheduling information
Pending I/O information

6. Process Control Block (PCB)
Process ID (PID)
This is an abstract view
Windows implementation of PCB is split in multiple data structures
Parent PID …
Next Process Block
PCB
List of open files
Handle Table
Image File Name
Thread Control Block (TCB)
List of Thread Control Blocks
Next TCB
Program Counter …
Registers …

7. CPU Switch from Thread to Thread
Interrupt or system call
executing
Save state into TCB1
ready or waiting
Reload state from TCB2
Interrupt or system call
ready or waiting
executing
Save state into TCB2
From a processor’s point of view, it executes instructions load from memory dictated by the changing values of the program counter. Over time, the program counter may refer to code in different programs. Each of these programs in execution forms a process with one or multiple threads.
When a thread whose state was “running” is interrupted, the current values of program counter and processor registers are saved in the corresponding thread control block and the the state of the thread is changed to “waiting” or “ready”. The OS is now free to put some other thread in the running state. The program counter and context data for this thread are loaded into the processor register and the thread begins execution.
There might be extra processing required if a context switch happens between threads in different processes. In this case, the mapping of virtual to physical addresses has to updated - which means that the memory management unit (MMU) has to be re-programmed.
Reload state from TCB1
ready or waiting
executing

8. Context Switch
When CPU switches to another thread, the system must save the state of the old thread and load the saved state for the new thread
Context-switch time is overhead; the system does no useful work while switching
Thread context-switching can be implemented in kernel or user mode
Interaction with memory management (MMU) is required when switching between threads in different processes

9. Thread Scheduling Queues
Ready queue
Maintains set of all threads ready and waiting to execute
There might be multiple ready queues, sorted by priorities
Device queue
Maintains set of threads waiting for an I/O device
There might be multiple queues for different devices
Threads migrate between the various queues

10. Ready Queue and I/O Device Queues
Time-out
Ready queue
CPU
Dispatch
Release
I/O 1 queue
I/O 1 wait
I/O 2 wait
I/O n wait
I/O occurs
I/O n queue

11. Optimization Criteria
CPU scheduling uses heuristics to manage the tradeoffs among contradicting optimization criteria.
Schedulers are optimized for certain workloads
Interactive vs. batch processing
I/O-intense vs. compute-intense
Common optimization criteria:
Maximize CPU utilization
Maximize throughput
Minimize turnaround time
Minimize waiting time
Minimize response time

12. Basic Scheduling Considerations
What invokes the scheduler?
Which assumptions should a scheduler rely on?
What are its optimization goals?
Rationale:
Multiprogramming maximizes CPU utilization
Thread execution experiences cycles of compute- and I/O-bursts
Scheduler should consider CPU burst distribution

13. Alternating Sequence of CPU and I/O Bursts…
load val
inc val
read file
CPU burst
Threads can be described as either:
I/O-bound – spends more time doing I/O than computations, many short CPU bursts
CPU-bound – spends more time doing computations; few very long CPU bursts
wait for I/O
I/O burst
inc count
add data, val
write file
CPU burst
wait for I/O
I/O burst
load val
inc val
read from file
CPU burst
wait for I/O
I/O burst …

14. Histogram of CPU-burst Times
distribution 10 20 30
Burst duration (msec)
Many short CPU bursts are typical
Exact figures vary greatly by process and computer

15. Schedulers Long-term scheduler (or job scheduler)
Selects which processes with their threads should be brought into the ready queue
Takes memory management into consideration (swapped-out processes)
Controls degree of multiprogramming
Invoked infrequently, may be slow
Short-term scheduler (or CPU scheduler)
Selects which threads should be executed next and allocates CPU
Invoked frequently, must be fast
Windows has no dedicated long-term scheduler

16. CPU Scheduler
Selects from among the threads in memory that are ready to execute, and allocates the CPU to one of them
CPU scheduling decisions may take place when a thread:
1. Switches from running to waiting state
2. Switches from running to ready state
3. Switches from waiting to ready
4. Terminates
Scheduling under 1 and 4 is nonpreemptive
All other scheduling is preemptive

17. Dispatcher
Dispatcher module gives control of the CPU to the thread selected by the short-term scheduler; this involves:
switching context
switching to user mode
jumping to the proper location in the user program to restart that program
Dispatch latency – time it takes for the dispatcher to stop one thread and start another running.
Windows scheduling is event-driven
No central dispatcher module in the kernel

18. Scheduling Algorithms: First-In, First-Out (FIFO)
Also known as First-Come, First-Served (FCFS)
Thread Burst Time
T1 20
T2 5
T3 4
Suppose that the threads arrive in the order: T1 , T2 , T3
The Gantt Chart for the schedule is:
Waiting time for T1 = 0; T2 = 20; T3 = 25
Average waiting time: ( )/3 = 15
Convoy effect: short thread behind long threads experience long waiting time T1 T2 T3 20 25 29

19. FIFO Scheduling (Cont.)
Suppose that the threads arrive in the order
T2 , T3 , T1 .
The Gantt chart for the schedule is:
Waiting time for T1 = 9; T2 = 0; T3 = 5
Average waiting time: ( )/3 = 4.66
Much better than previous case T1 T3 T2 9 5 29

20. Scheduling Algorithms: Round Robin (RR)
Preemptive version of FIFO scheduling algorithm
Each thread gets a small unit of CPU time (time quantum), usually milliseconds
After this time has elapsed, the thread is preempted and added to the end of the ready queue
Each of n ready thread gets 1/n of the CPU time in chunks of at most quantum q time units at once
Of n ready threads, no one waits more than (n-1)q time units
Performance
q large  FIFO
q small  q must be large with respect to context switch, otherwise overhead is too high

21. Example of RR with Quantum = 10
Thread Burst Time
T1 23
T2 7
T3 38
T4 14
Assuming all threads have same priority, the Gantt chart is:
Round-Robin favors CPU-intense over I/O-intense threads
Priority-elevation after I/O completion can provide a compensation
Windows uses Round-Robin with a priority-elevation scheme T1 T2 T3 T4 10 17 27 37 47 57 61 64 74 82

22. Shorter quantum yields more context switches
Thread execution time: 15
quantum 20 15 10 1 10 15 1 14 10 15
Longer quantum yields shorter average turnaround times

23. Scheduling Algorithms: Priority Scheduling
A priority number (integer) is associated with each thread
The CPU is allocated to the thread with the highest priority
Preemptive
Non-preemptive

24. Priority Scheduling - Starvation
Starvation is a problem:
low priority threads may never execute
Solutions
Decreasing priority & aging: the Unix approach
Decrease priority of CPU-intense threads
Exponential averaging of CPU usage to slowly increase priority of blocked threads
2) Priority Elevation: the Windows/VMS approach
Increase priority of a thread on I/O completion
System gives starved threads an extra burst

25. Multilevel Queue Ready queue is partitioned into separate queues:
Real-time (system, multimedia)
Interactive
Queues may have different scheduling algorithm,
Real-Time – RR
Interactive – RR + priority-elevation + quantum stretching
Scheduling must be done between the queues
Fixed priority scheduling (i.e., serve all from real-time threads then from interactive)
Possibility of starvation
Time slice – each queue gets a certain amount of CPU time which it can schedule amongst its threads;
CPU reserves

26. Multilevel Queue Scheduling
High priority
Real-time system threads
Real-time user threads
System threads
Interactive user threads
background threads
Low priority
Windows uses strict Round-Robin for real-time threads
Priority-elevation can be disabled for non-RT threads

27. Process Creation
Parent process creates children processes, which create other processes, forming a tree of processes
Processes start with one initial thread
Resource sharing models
Parent and children share all resources
Children share subset of parent’s resources
Parent and child share no resources
Execution
Parent’s and children's’ threads execute concurrently
Parent waits until children terminate

28. Process Creation (Cont.)
How to set up an address space
Child can be duplicate of parent
Child may have a program loaded into it
UNIX example
fork() system call creates new process
exec() system call used after a fork to replace the process’ memory space with a new program
Windows example
CreateProcess() system call create new process and loads program for execution

29. Processes Tree on a UNIX System

30. Process Termination
Last thread inside a process executes last statement and returns control to operating system (exit)
Parent may receive return code (via wait)
Process’ resources are deallocated by operating system
Parent may terminate execution of children processes (kill)
Child has exceeded allocated resources
Task assigned to child is no longer required
Parent is exiting
Operating system typically does not allow child to continue if its parent terminates (depending on creation flags)
Cascading termination inside process groups

31. Single and Multithreaded Processes
code
data
files
code
data
files
registers
stack
registers
registers
registers
stack
stack
stack
Thread
Thread
Thread
Thread
single-threaded
multi-threaded

32. Benefits of Multithreading
Higher Responsiveness
Dedicated threads for handling user events
Simpler Resource Sharing
All threads in a process share same address space
Utilization of Multiprocessor Architectures
Multiple threads may run in parallel

33. User Threads Thread management within a user-level threads library
Process is still unit of CPU scheduling from OS kernel perspective
Examples
POSIX Pthreads
Mach C-threads
Solaris threads
Fibers on Windows

34. Kernel Threads Supported by the Kernel Examples
Thread is unit of CPU scheduling
Examples
Windows
Solaris
OSF/1
Linux - Tasks can act like threads by sharing kernel data structures

35. Multithreading Models
How are user-level threads mapped on kernel threads?
Many-to-One
Many user-mode threads are mapped on a single kernel thread
One-to-One
Each user-mode thread is represented by a separate kernel thread
Many-to-Many
A set of user-mode threads is being mapped on another set of kernel threads

36. Many-to-One Model
User Thread
User Thread
User Thread
Many user-level threads are mapped to a single kernel thread
Used on systems that do not support kernel threads
Example:
POSIX Pthreads
Mach C-Threads
Windows Fibers
Kernel thread

37. One-to-One Model Each user-level thread maps to kernel thread Examples
User Thread
User Thread
User Thread
Each user-level thread maps to kernel thread
Examples
Windows threads
OS/2 threads
Kernel thread
Kernel thread
Kernel thread

38. Many-to-Many Model
User Thread
User Thread
User Thread
Allows many user level threads to be mapped to many kernel threads.
Allows the operating system to create a sufficient number of kernel threads.
Example
Solaris 2
Kernel thread
Kernel thread

39. Problems with Multithreading
Semantics of fork()/exec() or CreateProcess() system calls
Coordinated termination
Signal handling
Global data, errno, error handling
Thread specific data
Reentrant vs. non-reentrant system calls

40. Pthreads
a POSIX standard (IEEE c) API for thread creation and synchronization
API specifies behavior of the thread library, not an implementation
Implemented on many UNIX operating systems
Services for Unix (SFU) implement PThreads on Windows

41. Further Reading
Abraham Silberschatz, Peter B. Galvin, Operating System Concepts, John Wiley & Sons, 6th Ed., 2003;
Chapter 4 - Processes
Chapter 5 - Threads
Chapter 6 - CPU Scheduling
Mark E. Russinovich and David A. Solomon, Microsoft Windows Internals, 4th Edition, Microsoft Press, 2004.
Chapter 6 - Processes, Thread, and Jobs (from pp. 289)