October 9, 2002 Gary Kimura Lecture #5 October 9, 2002 CSE451 Threads Autumn 2002 Gary Kimura Lecture #5 October 9, 2002 CSE 451 Introduction to Operating Systems

Today Finish off with processes Threads a versatile programming construct that supercharges up dreary old processes

Motivation for Threads An OS process includes numerous things: An address space (defining all the code and data) System resources and accounting information A “thread of control” defining where the process is currently executing (basically, the PC and registers) Creating a new process can be costly, because of all of the structures that must be allocated And communicating among processes is costly, because most communication goes through the OS However a multiple thread of control paradigm is a wonderful programming tool that we really want to support in some useful way

Parallel Programs Without Threads Suppose we want to build a parallel program to execute on a multiprocessor, or a web server to handle multiple simultaneous web requests. We need to: Create several processes that can execute in parallel Cause each to share data, by possibly mapping to the same address space Give each a starting address and initial set of parameters The OS will then schedule these processes, in parallel, on the various processors, and maybe not even know they are really part of the same job

Cost of Doing Parallel Programs It can be costly doing parallel programs in an OS that does not support multi-threaded processes. There is a cost with creating and coordinating all of the processes There is also a lot of duplication, because they all share the same address space, opened files, protection, etc So any support that the OS can give for doing multi-threaded programs is a win

Lightweight Processes Looking at the previous slides, we need to ask ourselves, What’s similar in all these processes? They all share the same code and data (address space) They share almost everything in the process (e.g., opened files) What don’t they share? Each has its own PC, registers, and stack pointer So the idea is to Separate the process (address space, accounting, etc.) from that of the minimal “thread of control” (PC, SP, registers)? And we get multi-threaded processes

Adding Threads to Processes Some newer OSs (Mach, NT, modern Unix) support two entities: The process, which defines the address space and general process attributes (such as opened files, etc.) The thread, which defines a sequential execution stream within a process A thread is bound to a single process. For each process, however, there may be many threads Sharing data between threads is cheap Threads are the unit of scheduling Processes are containers in which threads execute

Thread Design Space older UNIXes MS/DOS address space one thread/process one thread/process one process many processes thread Java Mach, NT, Chorus, Linux, … many threads/process many threads/process one process many processes

(old) Process address space 0x7FFFFFFF stack (dynamic allocated mem) SP heap (dynamic allocated mem) address space static data (data segment) code (text segment) PC 0x00000000

(new) Address space with threads thread 1 stack SP (T1) 0x7FFFFFFF thread 2 stack SP SP (T2) thread 3 stack SP (T3) address space heap (dynamic allocated mem) static data (data segment) 0x00000000 PC code (text segment) PC (T2) PC (T1) PC (T3)

Separation of Threads and Processes Separating threads and processes makes it easier to support multi-threaded applications Concurrency (multi-threading) is useful for: Improving program structure Handling concurrent events (e.g., web requests) building parallel programs So, multi-threading is useful even on a uniprocessor.

Kernel thread and user-level threads Who is responsible for creating/managing threads? Two answers, in general: the OS (kernel threads) thread creation and management requires system calls the user-level process (user-level threads) a library linked into the program manages the threads Why is user-level thread management possible? threads share the same address space therefore the thread manager doesn’t need to manipulate address spaces threads only differ in hardware contexts (roughly) PC, SP, registers these can be manipulated by the user-level process itself!

Kernel Threads Each thread is a kernel object It is a schedulable entity. It goes through the same state transitions as we saw for processes (ready, running, and waiting) There are some performance issues A thread cannot directly yield control to another thread in the same process Kernel threads may be overly general, in order to support needs of different users, languages, etc.

User-Level Threads User level threads is essentially a way to mimic multi threading in a user level library A user-level thread is managed entirely by the run-time system (user-level code that is linked to your program). Each thread is represented simply by a PC, registers, stack and a little control block, managed in the user’s address space. Creating a new thread, switching between threads, and synchronizing between threads can all be done without kernel involvement. It can be really fast provided the application is well behaved The original Windows running on DOS was done along this line

Academic wisdom Kernel threads can still be too expensive thread operations are all system calls OS must perform all of the usual argument checks but want them to be as fast as a procedure call! must maintain kernel state for each thread can place limit on # of simultaneous threads, typically ~1000 User-level threads are small and fast each thread is represented simply by a PC, registers, a stack, and a small thread control block (TCB) creating a thread, switching between threads, and synchronizing threads are done via procedure calls no kernel involvement is necessary! user-level thread operations can be 10-100x faster than kernel threads as a result

User-level Thread Limitations But, user-level threads aren’t perfect tradeoff, as with everything else User-level threads are invisible to the OS there is no integration with the OS As a result, the OS can make poor decisions scheduling a process with only idle threads blocking a process whose thread initiated I/O, even though the process has other threads that are ready to run unscheduling a process with a thread holding a lock Solving this requires coordination between the kernel and the user-level thread manager

Example Kernel Thread Interface Some of the more common kernel thread interface calls are: Create Thread – Add a new thread to the process Terminate Thread – Destroy an existing thread Impersonate Thread – This is particularly important for server applications where each thread in a process needs to perform the task in the context of the client A wait or synchronization call – This is allows threads to synchronize among themselves and the OS in general A set and query Thread Information – Provide an interface to modify and query a threads state/status.

User-Level threads Interface Some of the more common user level thread interface calls are: Create Thread – Add a new thread to the process Stop Thread – Allows a thread to block itself Start Thread – Allows another thread to start a blocked thread Yield Thread – Voluntarily gives up the CPU to another thread in the process Exit Thread – Terminates the calling thread

User-level thread implementation a thread scheduler determines when a thread runs it uses queues to keep track of what threads are doing just like the OS and processes but, implemented at user-level as a library run queue: threads currently running ready queue: threads ready to run wait queue: threads blocked for some reason maybe blocked on I/O, maybe blocked on a lock how can you prevent a thread from hogging the CPU? how did the OS handle this?

Preemptive vs. non-preemptive Strategy 1: force everybody to cooperate a thread willingly gives up the CPU by calling yield() yield() calls into the scheduler, which context switches to another ready thread what happens if a thread never calls yield()? Stategy 2: use preemption scheduler requests that a timer interrupt be delivered by the OS periodically usually delivered as a UNIX signal (man signal) signals are just like software interrupts, but delivered to user-level by the OS instead of delivered to OS by hardware at each timer interrupt, scheduler gains control and context switches as appropriate

Thread Data Structures To add threads to the system we need to split up the PCB. Thread ID Thread state (ready, running, waiting) PC, SP, registers Wait queue list entries Priority (in some systems) Authentication (in some systems) These fields go into a new structure called a Thread Control Block (TCB)

What’s the best thread approach? Choose either side, you’ll have plenty of company Personally I like kernel threads What do I like about Kernel threads: More robust than user-level threads Allow impersonation Easier to tune the OS CPU scheduler to handle multiple threads in a process A thread doing a wait on a kernel resource (like I/O) does not stop the process from running What about user-level threads A lot faster if programmed correctly Can be better tuned for the exact application Note that user-level threads can be done on any OS

Some Things to Remember Each thread shares everything with all the other threads in the process They can read/write the exact same variables, so they need to synchronize themselves They can access each other’s runtime stack, so be very careful if you communicate using runtime stack variables Each thread should be able to retrieve a unique thread id that it can use to access thread local storage Multi-threading is great, but use it wisely

Next time So we have all these entities that want to run on the machine. How do we schedule them to run?