1. COSC 3407: Operating Systems Lecture 5: Independent Vs. Cooperating Threads

2. This lecture… u Thread Creation u Why do we need to handle cooperating threads? u Atomic operations

3. Thread Creation u Thread “fork” – create a new thread, three arguments: – Pointer to application routine to execute (fcnPtr) – Pointer to arguments records (fcnArgPtr) – Size of stack to allocate u Fork for threads not the same as fork for processes Fork(fcnPtr, fcnArgPtr, StackSize) Appl. Routine to run args

4. Thread Creation u Thread fork implementation: – Sanity check arguments – check stack size not infinite, fcnPtr points to something well formed, i.e. a valid code – Enter kernel mode (sanity check again) – Allocate a stack and a new TCB and initialize its register fields. » In particular, the stack pointer is made to point at the stack, the PC return address is made to point at an OS (assembler) routine ThreadRoot, and two of the registers are initialized to fcnPtr and fcnArgPtr – Put the newly allocated TCB on the ready list (Runnable). This will cause it to eventually be dispatched by run_new_thread, and start running the routine ThreadRoot.

5. Thread Creation u ThreadRoot: – Do start-up housekeeping (e.g., record start time, accounting information). – Return to user mode. – Call fcnPtr(fcnArgPtr). – Do thread finish-up:call ThreadFinish. ThreadRoot fcnPtr A B

6. Thread Finish u ThreadFinish: – Put any threads waiting on the termination of this thread on the ready list. – Can’t deallocate thread yet, since we’re still running on its stack. Record thread as “waitingToBeDestroyed”. – Call run_new_thread to run another thread. ThreadHouseKeeping will examine waithingToBeDestroyed and deallocate the finished thread’s TCB and stack. run_new_thread() { newThread = PickNewThread(); switch(curThread, newThread); ThreadHouseKeeping(); } ThreadRoot ThreadFinish

7. Additional Details u Thread fork is not the same thing as UNIX “fork”. UNIX fork creates a new process, so it has to create a new address space, in addition to a new thread. u For now, don’t worry about how switching between different processes’ address spaces is done. u Thread fork is very much like an asynchronous procedure call – it means, go do this work, where the calling thread does not wait for the callee to complete.

8. Parent-Child relationship u Every thread (and/or Process) has a parentage – A “parent” is a thread that creates another thread – A child of a parent was created by that parent Typical process tree for Solaris system

9. Thread Join u What if thread wants to exit early? – ThreadFinish() and exit() are essentially the same procedure entered at user level u What if the calling thread needs to wait? – Thread Join – wait for a forked thread to finish. – Calling thread will be taken off run queue and placed on waiting queue

10. ThreadJoin() system call u Where is a logical place to store this wait queue? – On queue inside the TCB  Similar to wait() system call in UNIX – Lets parents wait for child processes Other State TCB 9 Link Registers Other State TCB 6 Link Registers Other State TCB 16 Link Registers Head Tail Termination Wait queue TCB tid

11. Use of Join for Traditional Procedure Call u Thus, a traditional procedure call is logically equivalent to doing a fork then immediately doing a join. u This is a normal procedure call (synchronous): A() { B(); } B() { } u The procedure A can also be implemented as: A’() { Thread t = new Thread; t->Fork(B); t->Join(); }

12. Synchronous/Asynchronous Procedure calls u Why not replace synchronous with asynchronous procedure calls everywhere? – Overhead: allocate TCB, allocate stack, startup overhead, context switch – Crashing of a thread kills the process

13. Multithreading u Multiple activities within the same process – Web server, database server u No protection! u Need coordination between threads – Web crawler: multiple threads going to different links; make sure that different threads do not return the same page as it will waste bandwidth and resources – File system – different threads don’t apply the same lock to two different users

14. Multiprocessing vs. Multiprogramming u Multiprocessing = multiple CPU u Multiprogramming = multiple jobs or processes u Definition of “run concurrently” – scheduler is free to run threads in any order (e.g., FIFO, random, etc.) u For example:

15. Multiprocessing vs. Multiprogramming u Dispatcher can choose to run each thread to completion, or time-slice in big chunks, or time slice so that each thread executes only one instruction at a time (simulating a multiprocessor, where each CPU operates in lockstep). u If the dispatcher can do any of the above, programs must work under all cases, for all interleavings. u So how can you know if your concurrent program works? Whether all interleavings will work?

16. Definitions u Independent threads: – No state shared with other threads – Deterministic – input state determines result – Reproducible – (input state can be recreated)I/O, memory, … – Scheduling order doesn’t matter u Cooperating threads: – Shared state – Non-deterministic – Non-reproducible u Non-reproducibility and non-determinism means that bugs can be intermittent. u This makes debugging really hard!

17. Interactions Complicate Debugging u Is any program truly independent? – Every process shares the file system, OS resources, network, etc – Extreme example: buggy device driver causes thread A to crash “independent thread” B u You probably don’t realize how much you depend on reproducibility: – Example: Evil C compiler » Modifies files behind your back by inserting errors into C program unless you insert debugging code – Example: Debugging statements can overrun stack u Non-deterministic errors are really difficult to find – Example: Memory layout of kernel+user programs » depends on scheduling, which depends on timer/other things » Original UNIX had a bunch of non-deterministic errors – Example: Something which does interesting I/O » User typing of letters used to help generate secure keys

18. Why allow cooperating threads? u People cooperate; and computers model people’s behavior, so computers at some level have to cooperate! 1. Share resources/information – One computer, many users – One bank balance, many ATMs – Embedded systems (ex: robot control) 2. Speedup – Overlap I/O and computation – UNIX file system does read ahead – Multiprocessors – chop up program into smaller pieces 3. Modularity – chop large problem up into simpler pieces. For example, to compile: gcc – cpp | cc1 | cc2 | as | ld – This makes the system easier to extend; you can replace the assembler without changing the loader.

19. Some simple concurrent programs u Most of the time, threads are working on separate data, so scheduling order doesn’t matter. Initially, y = 12 u What about? u What are the possible values for x after the above? u What are the possible values of x below? u Can’t say anything useful about a concurrent program without knowing what are the underlying indivisible operations! Thread A Thread B x = 1 y = 2 x = y + 1 y = y * 2 x = 1 x = 2

20. Atomic operations u What we want is some way of allowing a thread to perform a task without having other threads interfere with the task. u Atomic operation: an operation that always runs to completion, or not at all. – It is indivisible: it can’t be stopped in the middle, and its state can’t be modified by someone else during the operation. u On most machines, memory reference and assignment (i.e., load and store) of words are atomic. u Many instructions are not atomic. – For example, on most 32-bit architectures, double precision floating point store is not atomic; it involves two separate memory operations.

21. High-level Example: Web Server u Server must handle many requests u Non-cooperating version: serverLoop() { con = AcceptCon(); ProcessFork(ServiceWebPage(),con); } u What are some disadvantages of this technique?

22. Threaded Web Server u Now, use a single process u Multithreaded (cooperating) version: serverLoop() { connection = AcceptCon(); ThreadFork(ServiceWebPage(),connection); } u Looks almost the same, but has many advantages: – Can share file caches kept in memory, results of CGI scripts, other things – Threads are much cheaper to create than processes, so this has a lower per-request overhead u Question: would a user-level (say many-to-one) thread package make sense here? – When one request blocks on disk, all block…

23. Thread Pools u Problem with previous version: Unbounded Threads – When web-site becomes too popular – throughput sinks u Instead, allocate a bounded “pool” of threads, representing the maximum level of multiprogramming master() { allocThreads(slave,queue); while(TRUE) { con=AcceptCon(); Enqueue(queue,con); wakeUp(queue); } } slave(queue) { while(TRUE) { con=Dequeue(queue); if (con==null) sleepOn(queue); else ServiceWebPage(con); } } Master Thread Thread Pool queue