1. Operating System Concepts

2. Lecture Contents Motivation for threads Thread Concept
Similarity and Difference between Process and Thread
Advantages of threads
Implementation of Multithreading
Multithreading Models
Thread Library Concept
Multithreading Issues
Case Study
*Please follow Silberchatz, Galvin and Gagne (Chapter 4), 7th edition

3. Issues with Processes
So far we have discussed Process, Creation of new process concept and its execution; However there are some issues with processes.
Processes are EXPENSIVE ! (E.g. resources that are to be allocated with every process)
Hardware resources (memory), kernel resources (PCB)
Link establishment for IPC (unrelated processes/related processes)
Concurrency
Motivation ? .. A mechanism that provides an alternative of a process (so that a new process is not needed)

4. Analogy
Active Entity
Works simultaneously with others (many threads executing)
Requires coordination (if many parts of process are executing in parallel, then coordination among resources is required)
Resources: I/O devices,CPU etc

5. Process VS Thread Process are represented in two ways:
In RAM ? Physical representation..
In Execution context ? CPU registers..
OS represents all this information by using PCB
Separate PCB for every process..
PCB in case of multi-threading..

6. Threads within a same process share…
Multiple and concurrent threads within a process share:
Code
Data
PCB (Process Control Block)
For a thread creation, only need to allocate CPU context and Stack !!
Threads are in the same process address space

7. Single and Multithreaded Processes

8. How processes are similar to threads or different ?
A thread can be in a state, similar to process (new, ready, running, waiting, terminate)
Threads can also create other threads.
Difference
Processes execute only within their own address space, whereas threads of same process execute within address space of one major process
Synchronization issues ?

9. Advantages of thread
Responsiveness - allow a program to continue running even if part of it is blocked (e.g. In a browser, multiple threads are working simultaneously to increase responsiveness)
Resource sharing - threads share the memory and resources of the process (lower main memory requirement)
Economy - it is more economical to create and context-switch threads (because they share resources of the process to which they belong)
Parallelization (computation speedup)
Specialization (repeatedly executing same code)
IPC not required!
Scalability (utilizing multiprocessor architectures)
Data sharing in two cores.

10. Some other things..
Some of these may block from time to time. By decomposing such an application into multiple sequential threads that run in parallel, the programming model becomes simpler.
Any benefit to OS ? OS can assist multiple threads if it’s multithreaded
Making OS multithreaded is helpful because they can support multiple apps at same time and execute them on different CPUs.

12. Benefits of multithreading (Single CPU)..
Are threads useful in Single CPU architecture or when num of threads > num of CPU ?
If t_idle (disk reading e.g.) > t_cxt_switch time, then it makes sense that switching among threads is useful
t_cxt_switch < p_cxt_switch

13. Benefits of multithreading (Multi CPU system)..

14. THREAD
Thread- A piece of code in a process which runs concurrently with execution of other piece of codes within the same process (same address space)
No need to create a new process
Resources that thread need to update/allocate privately:
Thread ID
CPU Context (Program Counter (PC), register set)
Stack
Priority

15. Basic thread mechanism
How to support threads?
Must have data structure that allows to distinguish between process and thread
Goals
To identify threads and keep record of them like PCB
Mechanisms to create and manage threads
Mechanisms to allow threads to communicate with eachother

16. Thread Creation
Thread type data structure that contains all information specific to that thread e.g. Program counter, registers, stack etc
Fork (proc,args) {new type thread data structure is created}
Join(thread) {parent thread blocked until all executed in case of bunch of threads processing on some input array}

17. How multithreading is implemented ?
(with respect to benefits to applications and OS)
OS level threads are visible to OS scheduler
OS scheduler decides the execution/mapping of these OS level threads
Some of the OS (kernel) level threads are linked to user level threads (in a process) and some are made to run their own code.
For a user level thread to be executed, it must be RECOGNIZED (mapped to a kernel level thread and then OS scheduler will schedule it on underlying hardware).

19. Multithreading Models
Depending on this concept, we can have three types of multithreading models:
One to One model: (for each user level thread, one kernel level thread)

20. One-to-One Model Benefits Each user-level thread maps to kernel thread
OS sees/understands that app is multithreaded
Benefits
Since OS already provide mechanisms to support its threads, user threads can directly benefit from this.
True concurrency - Another thread will be able to run when a thread makes a blocking system call.
Drawback:
Creating a user thread requires creating a corresponding kernel thread. (Overhead)
Portability issues: User level threads is relying support from kernel level thread management so an app running in one environment may not be able to run on other

21. Many to one Model
Many user level threads mapped to one OS level thread
There’s a thread management library on user level that decides which user level thread should be mapped to THAT ONE OS level thread.
That user level thread will run only when its scheduled by OS scheduler
Totally portable (everything done at user level, no need of support from OS level)
OS doesn’t know whether the app is multithreaded or not, it just sees kernel level thread
Whole process can get blocked !

22. Many-to-Many Model (Hybrid Model)
Multiple user level threads multiplexed over a smaller or equal number of kernel threads
Allows the operating system to create a sufficient number of kernel threads
Allow hybrid multithreading !
Which of three models solaris model is built?

23. Difference between the two techniques…
Advantages of User level threads (process scope)
Management and scheduling of thread is done by threading library in user space, so user level threads don’t invoke kernel for doing these tasks
User level threads can run on any OS, no changes are required to do on underlying kernel (Portability)
Process wide thread management (only limited to one process)
Disadvantages
Using User Level threading strategy, a multithreaded application cannot take advantage of multiprocessor system. (different threads of a same process cannot run on different CPU’s because OS considers a process a single threaded program)
If a thread is blocked, all the threads within a process gets blocked.
some threading libraries translate blocking system calls into nonblocking system calls to address this problem
Scheduling pripority in kernel level threads

24. Advantages of Kernel level threads (system scope)
Since kernel threads use the kernel scheduler, different kernel threads can run on different CPUs (taking advantage of multiprocessor system)
If a thread within a process gets blocked (e.g. waiting for an I/O), the scheduler can schedule another thread
If a process has large number of threads, it will receive large fraction of underlying hardware CPU.
Disadvantages
Frequent invoking of kernel for management of threads

27. Examples User Threads Kernel Threads
Thread management done by user-level threads library
Three primary thread libraries:
POSIX Pthreads
Win32 threads
Java threads
Kernel Threads
Supported by the Kernel
Examples
Windows XP/2000
Solaris
Linux
POSIX: an acronym for Portable Operating System Interface, is a family of standards specified by the IEEE Computer Society for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix and other operating systems.
This standard defines a standard operating system interface and environment, including a command interpreter (or "shell"), and common utility programs to support applications portability at the source code level. 
Pthreads: Pthreads refers to the POSIX standard (IEEE c) defining an API for thread creation and synchronization. This is a specification for thread behavior, not an implementation.
Threads have been standardized by the IEEE Technical Committee on Operating Systems, POSIX (Portable Operating System Interface). This standard allows users to write portable multi-thread programs.

28. Multithreading Models
Based on the number of kernel threads allocated correspondingly to the user level threads, we have three multithreading models
Many-to-one Model
One-to-one Model
Many-to-Many Model

29. Thread Libraries
Thread library provides programmer with API for creating and managing threads
Two primary ways of implementing
Library entirely in user space
Kernel-level library supported by the OS -> invoking a function in the library results with a system call

30. Examples of threading Libraries
Pthreads
Win32
Java
A POSIX standard (IEEE c) API for thread creation and synchronization
API specifies behavior of the thread library, implementation is up to development of the library (i.e. the OS designers)
This standard has been defined to make the writing of PORTABLE threaded program possible
May be provided either as user-level or kernel-level.
Common in UNIX operating systems (Solaris, Linux, Mac OS X)

31. Threading Issues Thread cancellation Asynchronous or deferred
Signal handling
Thread pools
Thread-specific data
Scheduler activations

32. Thread Cancellation Terminating a thread before it has finished
Example: multiple threads concurrently searching through a database and one thread returns the result
Stopping a webpage from loading any further
Two general approaches:
Asynchronous cancellation terminates the target thread immediately – troublesome if it’s in middle of updating some data
Deferred cancellation allows the target thread to periodically check if it should be cancelled – checking a flag.
Can only be cancelled at defined points called cancellation points

33. Signal Handling
Signals are used in UNIX systems to notify a process that a particular event has occurred
Example: illegal memory access, division by zero
A signal handler is used to process signals
Signal is generated by particular event
Signal is delivered to a process
Signal is handled
In a multithreaded process, where should a signal be delivered?
Options:
Deliver the signal to the thread to which the signal applies
Deliver the signal to every thread in the process
Deliver the signal to certain threads in the process
Assign a specific thread to receive all signals for the process

34. Thread Pools Reason for the need of thread pool:
Time consumed to create a new thread and destroy the previous ones – leads to slow performance
Unlimited threads could exhaust system resources, such as CPU time or memory
Solution:
Thread Pools
Create a number of threads at process startup and place them in a pool where they await work
Advantages:
Usually slightly faster to service a request with an existing thread than create a new thread
Also allows an application to specify the number of threads it requires

36. Thread Specific Data Scheduler activation
Allows each thread to have its own copy of specific data
Example, in a transaction-processing system, service each transaction in a separate thread. Furthermore, each transaction might be assigned a unique identifier. To associate each thread with its unique identifier, use thread-specific data.
Scheduler activation
User-Level Threads: If a single user-level thread blocks, the operating system blocks the entire multithreaded process. Another limitation to a many-to-one thread mapping is that a process's threads cannot simultaneously execute on multiple processors. Scheduler activations attempt to address these limitations to user-level threads. A scheduler activation is a kernel thread that can notify a user-level threading library of events (e.g., a thread has blocked or a processor is available). This type of kernel thread is called a "scheduler activation,“ because the user-level threading library can perform thread-scheduling operations when "activated" by an event notification, sometimes called an upcall.

37. Case Study
Windows XP
Linux

38. Windows XP

39. Windows XP Threads

40. Linux Threads Linux refers to them as tasks rather than threads
Thread creation is done through clone() system call
clone() accepts arguments that specify which resources to share with the child process