1. Chapter 3 Threads and Multithreading

2. Threads Overview Multithreading Models Thread Scheduling
Threading Issues
P-threads
Solaris 2 Threads
Windows 2000 Threads
Linux Threads
Java Threads

3. Overview
A thread, sometimes called a lightweight process (LWP), is a basic unit of CPU utilization.
Unit of execution (unit of dispatching) and a collection of resources, with which the unit of execution is associated, characterize the notion of a process.
A thread is the abstraction of a unit of execution.
A thread is a single sequence stream within a process.

4. Overview
As a basic unit of CPU utilization, a thread consists of an instruction pointer (also referred to as the PC or instruction counter), a thread id, a CPU register set and a stack.
A thread shares its code and data, as well as system resources and other OS related information, with its peer group (other threads of the same process).
If the process has multiple threads of control, it can do more than one task at a time.

5. The Thread Model (a) Three processes each with one thread
(b) One process with three threads

6. Thread Example

7. Single and Multithreaded Processes

8. Motivation A Process with Several Threads of Control. Web Browser
Displaying context while Retrieving Data from the network.
Word Processor
A thread for displaying graphics, another thread for reading keystrokes, and a third thread for performing spelling checking.
An application performs several similar tasks efficiently.
Web Server
When several clients are accessing a web server, the server accepts a client request and creates a separate thread serving the request.
If separate processes are created instead, process creation and management overhead follows.
RPC
Serving several concurrent client requests.

9. A multithreaded Web server
Thread Usage
A multithreaded Web server

10. Thread Example (2) Word processor may have a thread for;
Displaying graphics.
Reading keystrokes from the user.
Performing spelling and grammar checking in background.

11. Benefits Responsiveness:
A multithreaded web browser could still allow user interaction in one thread while an image is being loaded in another thread.
Increasing Responsiveness to the user by running several threads of control.
Resource Sharing:
Several different threads can share the same memory and resources.
Economy:
Allocating memory and resources is costly. But, threads share resources to which they belong.
Context switching of threads is more economical.
Utilization of MP Architectures:
Each process may be running on a different processor.

12. Each thread has its own stack
The Thread Model
Each thread has its own stack

13. Multicore Programming
Multicore systems putting pressure on programmers, challenges include
Dividing activities
Balance
Data splitting
Data dependency
Testing and debugging

14. Multithreaded Server Architecture

15. Concurrent Execution on a Single-core System

16. Parallel Execution on a Multicore System

17. User Threads
Supported above the kernel and are implemented by thread library at the user level.
The library supports thread creating, scheduling, and management with no support from the kernel.
Kernel is unaware of user-level threads, all thread creation and scheduling are done in user space. Hence, they are generally fast.
User thread library includes:
- POSIX Pthreads, Mach C-threads, Solaris2 UI-threads

18. User Threads Advantages
User-level threads does not require modification to OS.
Simple Representation: Each thread is represented simply by a PC, registers, stack and a small control block, all stored in the user process address space.
Simple Management: Means that creating a thread, switching between threads and synchronization between threads can all be done without intervention of the kernel.
Fast and Efficient: Thread switching is not much more expensive than a procedure call.

19. User Threads Disadvantages
There is a lack of coordination between threads and operating system kernel. Therefore, process as whole gets one time slice irrespective of whether process has one thread or 1000 threads within. It is up to each thread to relinquish control to other threads.
Any user-level thread performing a blocking system call will cause the entire process to block, even if other threads are available to run.

20. Kernel Threads Supported directly by the OS.
Kernel performs thread creation, scheduling and management in kernel space.
Kernel threads are generally slower.
If a thread performs a blocking system call, the kernel can schedule another thread.
Examples
- Windows 95/98/NT/2000, Solaris, Tru64, UNIX, BeOS, Linux

21. Kernel Threads Advantages
Because kernel has full knowledge of all threads, Scheduler may decide to give more time to a process having large number of threads than process having small number of threads.
Kernel-level threads are especially good for applications that frequently block.
In multiprocessor system, kernel can schedule threads on different processors.

22. Kernel Threads Disadvantages
The kernel-level threads are slow and inefficient. For instance, threads operations are hundreds of times slower than that of user-level threads.
Since kernel must manage and schedule threads as well as processes. It require a full thread control block (TCB) for each thread to maintain information about threads. As a result there is significant overhead and increased in kernel complexity.

23. Multithreading Models
Many-to-One
Many user-level threads mapped to single kernel thread.
Thread management is done in user space, and so it is efficient.
Only one thread can access the kernel at a time, multiple threads are unable to run in parallel on multiprocessors.
User level threads libraries implemented on OS that do not support kernel threads.
Green Thread – library available for Solaris 2.

24. Many-to-One Model

25. Multithreading Models (Contd)
One-to-One
Each user-level thread maps to kernel thread.
Provides concurrency than many-to-one by allowing another thread to run when a thread makes a blocking system call.
Allow multiple threads to run in parallel on multiprocessor.
Drawback:
Creating a user thread requires creating the corresponding kernel thread.
Overhead of Thread Creation and Management
Examples
- Windows 95/98/NT/2000, OS/2

26. One-to-one Model

27. Many-to-Many Model
Multiplexes many user level threads to a smaller or equal number of kernel threads.
The model suffers from neither of those shortcomings of many-to-one and one-to-one models.
No Concurrency, Expensive Threads
Allows the operating system to create a sufficient number of kernel threads.
Example:
Solaris 2, Windows NT/2000 with the ThreadFiber package

28. Many-to-Many Model

29. Thread Scheduling
There are two main approaches to the scheduling of threads.
Preemptive scheduling:
Thread may be suspended to make way for another thread, even it is in run able condition.
Non-preemptive scheduling:
Coroutine scheduling, thread runs until it makes a call that causes it to be rescheduled and another thread to be run.
Both have been implemented for the C threads package; the programmer chooses one or other by linking in different libraries.

30. Threading Issues There are six(6) Threading Issues
Semantics of fork() and exec() system calls.
Thread cancellation.
Signal handling
Thread pools
Thread specific data
Scheduler activations

31. Threading Issues (Contd)
Semantics of fork() and exec() system calls.
If one thread calls fork( ), does the new process duplicate all threads or is the new process single-threaded?
Some UNIX systems provide two versions of fork( ).
One that duplicates all threads.
Another that duplicates only the thread that invoked the fork() system call.
If a thread invokes the exec() system call, the program specified the parameter to exec() will replace the entire process, including all threads

32. Threading Issues (Contd)
Thread cancellation.
A task of terminating a (target) thread before it has completed.
Example: If multiple threads are concurrently searching through a database and one thread returns the expected result, the remaining threads might be cancelled.
Example: When a user press the stop button on a web browser, the thread loading a web page should be cancelled.
Asynchronous Cancellation
One thread immediately terminates the target thread.
Deferred Cancellation
The target thread can periodically check if it should terminate.
Pthreads API provides deferred cancellation.

33. Threading Issues (Contd)
Signal handling.
A signal is used in UNIX systems to notify a process that a particular event has occurred.
Synchronous Signal Delivery.
Synchronous signals are delivered to the same process that performed the operation causing the signal.
Example: An illegal memory access or division by zero
Asynchronous Signal Delivery
When a signal is generated by en event external to a running process, the signal is asynchronously delivered typically to another process.
Example: Terminating a process with <control> <C> or having a timer expire.

34. Threading Issues (Contd)
Signal handling (Contd).
Three Steps of Signal Processing.
A signal is generated by the occurrence of an event.
A generated signal is delivered to a process.
Once delivered, the signal must be handled.
Two Ways of Signal Handling.
A Default Signal Handler (run by kernel).
A User-defined Signal Handler
Overwrites the default handler.

35. Threading Issues (Contd)
Signal handling (Contd).
Options to delivering signal for Multi-threading.
Deliver it to the thread to which the signal applies.
Deliver it to every thread in the process.
Deliver it to certain threads in the process.
Deliver it to a specific thread which was assigned to receive all signals for the process.
The method for delivering a signal depends upon the type of signal generated.
Ex: Synchronous signal
Ex: Asynchronous signal terminating a process

36. Threading Issues (Contd)
Thread pools.
Create a number of threads in a pool where they await work
Motivation
A multi-threaded server may spend too much time in creating and deleting many threads.
A large number of (unlimited) threads could exhaust system resource such as CPU time and memory.
Solution
To create a number threads at process startup and place them into a pool, where they sit and wait for job.
When a server receives a request, it awakens a thread from the pool. And, the thread returns to poll upon completing the job.
If no available thread in the pool, the server waits until one becomes available.

37. Threading Issues (Contd)
Thread pools (Contd).
Benefits
It is faster to service a request with an existing thread than waiting to create a new thread.
A thread pool limits the number of threads that exist at any one point.
Size of the thread pool
Can be static or dynamic.

38. Threading Issues (Contd)
Thread pools (Contd).
Advantages:
Usually slightly faster to service a request with an existing thread than create a new thread
Allows the number of threads in the application(s) to be bound to the size of the pool 38

39. Threading Issues (Contd)
Thread - Specific Specific Data.
Threads share the data of the process.
One of the benefits of multi-threaded programming
However, in some cases, each thread needs its own copy of certain data, called thread-specific data.
Ex: Transaction Processing (TP) System
Each transaction is serviced by a separate thread, and the transaction has additional data such as unique transaction ID and status, i.e. thread-specific data.
Most thread library including Win32 and Pthreads support this data.

40. Scheduler Activations
Both M:M and Two-level models require communication to maintain the appropriate number of kernel threads allocated to the application
Scheduler activations provide upcalls - a communication mechanism from the kernel to the thread library
This communication allows an application to maintain the correct number kernel threads 40

41. Thread Libraries
Thread library provides programmer with API for creating and managing threads
Two primary ways of implementing
Library entirely in user space
Kernel-level library supported by the OS

42. Pthreads
A POSIX standard (IEEE c) API for thread creation and synchronization.
API specifies behavior of the thread library, implementation is up to development of the library.
Common in UNIX operating systems.
Windows operating systems have generally not supported Pthreads.

43. Solaris 2 Threads Solaris 2 is a version of UNIX. It supports
Kernel Threads,
User level Threads, and
An intermediate level of threads, lightweight process (LWP).
Each process contains at least one LWP. The thread library multiplexes user-level threads on the pool of LWP, and only the user-level threads currently connected to an LWP accomplish work.
Each LWP has a kernel thread, and some kernel threads have no associated LWP.
Many-to-Many Model

44. Solaris 2 Threads

45. Solaris 2 Threads Data Structure for Implementation
User level thread contains a thread ID, register set, stack, and priority.
An LWP has a register set for the user-level thread it is running, as well as memory and accounting information. An LWP is a kernel data structure.
A kernel thread has only a small data structure and a stack. The data structure includes a copy of kernel registers, a pointer to LWP to which it is attached, and priority and scheduling information.
Solaris Process
Has a conventional PCB.

46. Solaris Process