Chapter 3 Threads and Multithreading

Threads Overview Multithreading Models Thread Scheduling Threading Issues P-threads Solaris 2 Threads Windows 2000 Threads Linux Threads Java Threads

Overview A thread, sometimes called a lightweight process (LWP), is a basic unit of CPU utilization. Unit of execution (unit of dispatching) and a collection of resources, with which the unit of execution is associated, characterize the notion of a process. A thread is the abstraction of a unit of execution. A thread is a single sequence stream within a process.

Overview As a basic unit of CPU utilization, a thread consists of an instruction pointer (also referred to as the PC or instruction counter), a thread id, a CPU register set and a stack. A thread shares its code and data, as well as system resources and other OS related information, with its peer group (other threads of the same process). If the process has multiple threads of control, it can do more than one task at a time.

The Thread Model (a) Three processes each with one thread (b) One process with three threads

Thread Example

Single and Multithreaded Processes

Motivation A Process with Several Threads of Control. Web Browser Displaying context while Retrieving Data from the network. Word Processor A thread for displaying graphics, another thread for reading keystrokes, and a third thread for performing spelling checking. An application performs several similar tasks efficiently. Web Server When several clients are accessing a web server, the server accepts a client request and creates a separate thread serving the request. If separate processes are created instead, process creation and management overhead follows. RPC Serving several concurrent client requests.

A multithreaded Web server Thread Usage A multithreaded Web server

Thread Example (2) Word processor may have a thread for; Displaying graphics. Reading keystrokes from the user. Performing spelling and grammar checking in background.

Benefits Responsiveness: A multithreaded web browser could still allow user interaction in one thread while an image is being loaded in another thread. Increasing Responsiveness to the user by running several threads of control. Resource Sharing: Several different threads can share the same memory and resources. Economy: Allocating memory and resources is costly. But, threads share resources to which they belong. Context switching of threads is more economical. Utilization of MP Architectures: Each process may be running on a different processor.

Each thread has its own stack The Thread Model Each thread has its own stack

Multicore Programming Multicore systems putting pressure on programmers, challenges include Dividing activities Balance Data splitting Data dependency Testing and debugging

Multithreaded Server Architecture

Concurrent Execution on a Single-core System

Parallel Execution on a Multicore System

User Threads Supported above the kernel and are implemented by thread library at the user level. The library supports thread creating, scheduling, and management with no support from the kernel. Kernel is unaware of user-level threads, all thread creation and scheduling are done in user space. Hence, they are generally fast. User thread library includes: - POSIX Pthreads, Mach C-threads, Solaris2 UI-threads

User Threads Advantages User-level threads does not require modification to OS. Simple Representation: Each thread is represented simply by a PC, registers, stack and a small control block, all stored in the user process address space. Simple Management: Means that creating a thread, switching between threads and synchronization between threads can all be done without intervention of the kernel. Fast and Efficient: Thread switching is not much more expensive than a procedure call.

User Threads Disadvantages There is a lack of coordination between threads and operating system kernel. Therefore, process as whole gets one time slice irrespective of whether process has one thread or 1000 threads within. It is up to each thread to relinquish control to other threads. Any user-level thread performing a blocking system call will cause the entire process to block, even if other threads are available to run.

Kernel Threads Supported directly by the OS. Kernel performs thread creation, scheduling and management in kernel space. Kernel threads are generally slower. If a thread performs a blocking system call, the kernel can schedule another thread. Examples - Windows 95/98/NT/2000, Solaris, Tru64, UNIX, BeOS, Linux

Kernel Threads Advantages Because kernel has full knowledge of all threads, Scheduler may decide to give more time to a process having large number of threads than process having small number of threads. Kernel-level threads are especially good for applications that frequently block. In multiprocessor system, kernel can schedule threads on different processors.

Kernel Threads Disadvantages The kernel-level threads are slow and inefficient. For instance, threads operations are hundreds of times slower than that of user-level threads. Since kernel must manage and schedule threads as well as processes. It require a full thread control block (TCB) for each thread to maintain information about threads. As a result there is significant overhead and increased in kernel complexity.

Multithreading Models Many-to-One Many user-level threads mapped to single kernel thread. Thread management is done in user space, and so it is efficient. Only one thread can access the kernel at a time, multiple threads are unable to run in parallel on multiprocessors. User level threads libraries implemented on OS that do not support kernel threads. Green Thread – library available for Solaris 2.

Many-to-One Model

Multithreading Models (Contd) One-to-One Each user-level thread maps to kernel thread. Provides concurrency than many-to-one by allowing another thread to run when a thread makes a blocking system call. Allow multiple threads to run in parallel on multiprocessor. Drawback: Creating a user thread requires creating the corresponding kernel thread. Overhead of Thread Creation and Management Examples - Windows 95/98/NT/2000, OS/2

One-to-one Model

Many-to-Many Model Multiplexes many user level threads to a smaller or equal number of kernel threads. The model suffers from neither of those shortcomings of many-to-one and one-to-one models. No Concurrency, Expensive Threads Allows the operating system to create a sufficient number of kernel threads. Example: Solaris 2, Windows NT/2000 with the ThreadFiber package

Many-to-Many Model

Thread Scheduling There are two main approaches to the scheduling of threads. Preemptive scheduling: Thread may be suspended to make way for another thread, even it is in run able condition. Non-preemptive scheduling: Coroutine scheduling, thread runs until it makes a call that causes it to be rescheduled and another thread to be run. Both have been implemented for the C threads package; the programmer chooses one or other by linking in different libraries.

Threading Issues There are six(6) Threading Issues Semantics of fork() and exec() system calls. Thread cancellation. Signal handling Thread pools Thread specific data Scheduler activations

Threading Issues (Contd) Semantics of fork() and exec() system calls. If one thread calls fork( ), does the new process duplicate all threads or is the new process single-threaded? Some UNIX systems provide two versions of fork( ). One that duplicates all threads. Another that duplicates only the thread that invoked the fork() system call. If a thread invokes the exec() system call, the program specified the parameter to exec() will replace the entire process, including all threads

Threading Issues (Contd) Thread cancellation. A task of terminating a (target) thread before it has completed. Example: If multiple threads are concurrently searching through a database and one thread returns the expected result, the remaining threads might be cancelled. Example: When a user press the stop button on a web browser, the thread loading a web page should be cancelled. Asynchronous Cancellation One thread immediately terminates the target thread. Deferred Cancellation The target thread can periodically check if it should terminate. Pthreads API provides deferred cancellation.

Threading Issues (Contd) Signal handling. A signal is used in UNIX systems to notify a process that a particular event has occurred. Synchronous Signal Delivery. Synchronous signals are delivered to the same process that performed the operation causing the signal. Example: An illegal memory access or division by zero Asynchronous Signal Delivery When a signal is generated by en event external to a running process, the signal is asynchronously delivered typically to another process. Example: Terminating a process with <control> <C> or having a timer expire.

Threading Issues (Contd) Signal handling (Contd). Three Steps of Signal Processing. A signal is generated by the occurrence of an event. A generated signal is delivered to a process. Once delivered, the signal must be handled. Two Ways of Signal Handling. A Default Signal Handler (run by kernel). A User-defined Signal Handler Overwrites the default handler.

Threading Issues (Contd) Signal handling (Contd). Options to delivering signal for Multi-threading. Deliver it to the thread to which the signal applies. Deliver it to every thread in the process. Deliver it to certain threads in the process. Deliver it to a specific thread which was assigned to receive all signals for the process. The method for delivering a signal depends upon the type of signal generated. Ex: Synchronous signal Ex: Asynchronous signal terminating a process

Threading Issues (Contd) Thread pools. Create a number of threads in a pool where they await work Motivation A multi-threaded server may spend too much time in creating and deleting many threads. A large number of (unlimited) threads could exhaust system resource such as CPU time and memory. Solution To create a number threads at process startup and place them into a pool, where they sit and wait for job. When a server receives a request, it awakens a thread from the pool. And, the thread returns to poll upon completing the job. If no available thread in the pool, the server waits until one becomes available.

Threading Issues (Contd) Thread pools (Contd). Benefits It is faster to service a request with an existing thread than waiting to create a new thread. A thread pool limits the number of threads that exist at any one point. Size of the thread pool Can be static or dynamic.

Threading Issues (Contd) Thread pools (Contd). Advantages: Usually slightly faster to service a request with an existing thread than create a new thread Allows the number of threads in the application(s) to be bound to the size of the pool 38

Threading Issues (Contd) Thread - Specific Specific Data. Threads share the data of the process. One of the benefits of multi-threaded programming However, in some cases, each thread needs its own copy of certain data, called thread-specific data. Ex: Transaction Processing (TP) System Each transaction is serviced by a separate thread, and the transaction has additional data such as unique transaction ID and status, i.e. thread-specific data. Most thread library including Win32 and Pthreads support this data.

Scheduler Activations Both M:M and Two-level models require communication to maintain the appropriate number of kernel threads allocated to the application Scheduler activations provide upcalls - a communication mechanism from the kernel to the thread library This communication allows an application to maintain the correct number kernel threads 40

Thread Libraries Thread library provides programmer with API for creating and managing threads Two primary ways of implementing Library entirely in user space Kernel-level library supported by the OS

Pthreads A POSIX standard (IEEE 1003.1c) API for thread creation and synchronization. API specifies behavior of the thread library, implementation is up to development of the library. Common in UNIX operating systems. Windows operating systems have generally not supported Pthreads.

Solaris 2 Threads Solaris 2 is a version of UNIX. It supports Kernel Threads, User level Threads, and An intermediate level of threads, lightweight process (LWP). Each process contains at least one LWP. The thread library multiplexes user-level threads on the pool of LWP, and only the user-level threads currently connected to an LWP accomplish work. Each LWP has a kernel thread, and some kernel threads have no associated LWP. Many-to-Many Model

Solaris 2 Threads

Solaris 2 Threads Data Structure for Implementation User level thread contains a thread ID, register set, stack, and priority. An LWP has a register set for the user-level thread it is running, as well as memory and accounting information. An LWP is a kernel data structure. A kernel thread has only a small data structure and a stack. The data structure includes a copy of kernel registers, a pointer to LWP to which it is attached, and priority and scheduling information. Solaris Process Has a conventional PCB.

Solaris Process