1. CS 2200 Presentation 19 Threads

2. Test Preview A misguided java programmer is looking at the specs for a computer. It's a byte addressed 32-bit model equipped with a cache. The memory reference is divided like this: tag 19, index 7, offset 6. How many lines are in the cache? 1.64 2.128 3.1/2 Meg 4.None of the above

3. Questions?

4. Our Road Map Processor Networking Parallel Systems I/O Subsystem Memory Hierarchy

5. Recall Process –Program Counter –Registers –Stack –Code (Text) –Data –Page Table –etc. Processes must be protected from one another. Memory

6. Recall Context Switching –Requires considerable work What about a single users application? –Is there a way to make it more efficient –In effect, allow the user to have multiple processes executing in the same space? –Yes, solution: Threads or Multithreading

7. What is Multithreading? Technique allowing program to do multiple tasks –Example: Java GUI's

8. Java Event Handling import java.awt.*; public class HelloGUI { public static void main (String[ ] arg) { System.out.println (“About to make GUI”); Frame f = new Frame (“Hello GUIs”); f.setSize( 200, 200 ); f.show(); System.out.println (“Finished making GUI”); }// main }// class HelloGUI callback Our event handling code The code trapping this event appears in the graphics thread

9. What is Multithreading? Technique allowing program to do multiple tasks –Example: Java GUI's Is it a new technique? –has existed since the 70’s (concurrent Pascal, Ada tasks, etc.) Why now? –“Time has come for this technology” –Emergence of SMP’s in particular

10. SMP? What is an SMP? –Multiple CPUs in a single box sharing all the resources such as memory and I/O Is a dual-processor SMP more cost effective than two uniprocessor boxes? –Yes, (roughly 20% more for a dual processor SMP compared to a uniprocessor). –Modest speedup for a program on a dual- processor SMP over a uniprocessor will make it worthwhile.

11. What is a Thread? Basic unit of CPU utilization A lightweight process (LWP) Consists of –Program Counter –Register Set –Stack Space Shares with peer threads –Code –Data –OS Resources Open files Signals

12. Threads Can be context switched more easily –Registers and PC –Not memory management Can run on different processors concurrently in an SMP Share CPU in a uniprocessor May (Will) require concurrency control programming like mutex locks.

13. active Process Threads in a Uniprocessor? Allows concurrency between I/O and user processing even in a uniprocessor box

14. Threads and OS Traditional OS DOS –Memory layout user kernel Data –Protection between user and kernel? Data Program DOS Code

15. User Kernel Kernel code and data –Protection between user and kernel? Process code and data PCB P1 Process code and data PCB P2 Threads and OS Unix - memory layout

16. Threads and OS Programs in a traditional OS are single threaded –One PC per program (process), one stack, one set of CPU registers –If a process blocks (say disk I/O, network communication, etc.) then no progress for the program as a whole

17. MultiThreaded Operating Systems How widespread is support for threads in OS? –Digital Unix, Sun Solaris, Win9x, Win NT, Win2k, Linux, Free BSD, etc Process vs. Thread? –In a single threaded program, the state of the executing program is contained in a process –In a multithreaded program, the state of the executing program is contained in several ‘concurrent’ threads

18. What is this? 1.Crack in slide 2.Hair stuck on lens 3.Symbol for thread 4.San Andreas Fault

19. Process Vs. Thread user kernel Kernel code and data t1 –Computational state (PC, regs, …) for each thread –How different from process state? codedatacodedata t2t3t1 PCB P1P2

20. Memory Layout Multithreaded program has a per-thread stack Heap, static, and code are common to all threads MT program stk1 stk2 stk3... Heap Static Code ST program stack1 Heap Static Code

21. Threads Threaded code different from non- threaded? –Protection for data shared among threads Mutex –Synchronization among threads Way for threads to talk (signal) to one another –Thread-safe libraries NOTES strtok is unsafe for multi-thread applications. strtok_r is MT-Safe and should be used instead.

22. Typical Operation Main programs creates (or spawns) threads Threads may –perform one task and die –last for duration of program Threads must –be able to synchronize activity –communicate with one another

23. Programming Support for Threads Creation –pthread_create(top-level procedure, args) Termination –return to top-level procedure –explicit cancel Rendezvous –creator can wait for children pthread_join(child_tid) Synchronization –pthread_mutex_... –condition variables (pthread_cond_...)

24. Programming with Threads Synchronization –For coordination of the threads Communication –For inter-thread sharing of data –Threads can be in different processors –How to achieve sharing in SMP? Software: accomplished by keeping all threads in the same address space by the OS Hardware: accomplished by hardware shared memory and coherent caches

25. Synchronization Primitives lock and unlock –mutual exclusion among threads –busy-waiting vs. blocking –pthread_mutex_trylock: no blocking (rare) –pthread_mutex_lock: blocking –pthread_mutex_unlock condition variables –pthread_cond_wait: block for a signal –pthread_cond_signal: signal one waiting thread –pthread_cond_broadcast: signal all waiting threads

26. Example lock(mutex); while (resource_state == BUSY) //spin; resource_state = BUSY; unlock(mutex); use resource; lock(mutex); resource_state = FREE; unlock(mutex); Initially mutex is unlocked resource_state is FREE Will this work? 1 - Yes 2 - No 3 - Maybe

27. Example lock(mutex); while (resource_state == BUSY) //spin; resource_state = BUSY; unlock(mutex); use resource; lock(mutex); resource_state = FREE; unlock(mutex); Thread 1

28. Example lock(mutex); while (resource_state == BUSY) //spin; resource_state = BUSY; unlock(mutex); use resource; lock(mutex); resource_state = FREE; unlock(mutex); Thread 1 Thread 2

29. Example lock(mutex); while (resource_state == BUSY) //spin; resource_state = BUSY; unlock(mutex); use resource; lock(mutex); resource_state = FREE; unlock(mutex); Thread 1 Thread 2

30. Example with cond-var lock(mutex) while(resource_state == BUSY) wait(cond_var); /* implicitly give up mutex */ /* implicitly re-acquire mutex */ resource_state = BUSY unlock(mutex) /* use resource */ lock(mutex) resource_state = FREE; unlock(mutex) signal(cond_var);

31. Example with cond-var lock(mutex) while(resource_state == BUSY) wait(cond_var); /* implicitly give up mutex */ /* implicitly re-acquire mutex */ resource_state = BUSY unlock(mutex) /* use resource */ lock(mutex) resource_state = FREE; unlock(mutex) signal(cond_var); T1

32. Example with cond-var lock(mutex) while(resource_state == BUSY) wait(cond_var); /* implicitly give up mutex */ /* implicitly re-acquire mutex */ resource_state = BUSY unlock(mutex) /* use resource */ lock(mutex) resource_state = FREE; unlock(mutex) signal(cond_var); T1 T2

33. pthreads Mutex –Must create mutex variables pthread_mutex_t padlock; –Must initialize mutex variable pthread_mutex_init(&padlock, NULL); Condition Variable (used for signaling) –Must create condition variables pthread_cond_t non_full; –Must initialize condition variables pthread_cond_init(&non_full, NULL);

34. Classic CS Problem Producer Consumer Producer If (! full) Add item to buffer empty = FALSE if(buffer_is_full) full = TRUE Consumer If (! empty) Remove item from buffer full = FALSE if(buffer_is_empty) empty = TRUE full empty buffer...

35. Example Producer Threads Program while(forever){ // produce item pthread_mutex_lock(padlock); while (full) pthread_cond_wait(non_full, padlock); // add item to buffer; buffercount++; if (buffercount == BUFFERSIZE) full = TRUE; empty = FALSE; pthread_mutex_unlock(padlock); pthread_cond_signal(non_empty); }

36. Example Consumer Threads Program while(forever) { pthread_mutex_lock(padlock); while (empty) pthread_cond_wait (non_empty, padlock); // remove item from buffer; buffercount--; full = false; if (buffercount == 0) empty = true; pthread_mutex_unlock(padlock); pthread_cond_signal(non_full); // consume_item; }

37. // Producer while(forever){ // produce item pthread_mutex_lock(padlock); while (full) pthread_cond_wait(non_full, padlock); // add item to buffer; buffercount++; if (buffercount == BUFFERSIZE) full = TRUE; empty = FALSE; pthread_mutex_unlock(padlock); pthread_cond_signal(non_empty); } // Consumer while(forever) { pthread_mutex_lock(padlock); while (empty) pthread_cond_wait (non_empty, padlock); // remove item from buffer; buffercount--; full = false; if (buffercount == 0) empty = true; pthread_mutex_unlock(padlock); pthread_cond_signal(non_full); // consume_item; }

38. Using Threads Servers dispatcher model Mailbox dispatcher Workers

39. Team model Pipelined model Mailbox

40. Threads Implementation User level threads –OS independent –Scheduler is part of the runtime system –Thread switch is cheap (save PC, SP, regs) –Scheduling customizable, i.e., more application control –Blocking call by thread blocks process

41. Solution to blocking problem in user level threads –Non-blocking version of all system calls Switching among user level threads –Yield voluntarily –How to make preemptive? Timer interrupt from kernel to switch

42. Kernel Level –Expensive thread switch –Makes sense for blocking calls by threads –Kernel becomes complicated: process vs. threads scheduling –Thread packages become non-portable Problems common to user and kernel level threads –Libraries –Solution is to have thread-safe wrappers to such library calls

43. Questions?

44. Solaris Threads Three kinds –user, lwp, kernel User: Any number can be created and attached to lwp’s One to one mapping between lwp and kernel threads Kernel threads known to the OS scheduler If a kernel thread blocks, associated lwp, and user level threads block as well

45. Solaris Terminology

46. More Conventional Terminology Thread kernel thread (user-level view) (Inside the kernel) Processes P1P2P3

47. Questions? Does kernel see user threads? Are all threads from P3 on same CPU? Are all threads in kernel attached to lwp’s? If the left-most thread of P3 issues a blocking system call does P3 block? Who schedules lwp’s? Who schedules threads?

48. Kernel Threads vs. User Threads Advantages of kernel threads –Can be scheduled on multiple CPU’s –Can be preempted by CPU –Kernel scheduler knows their relative priorities Advantages of user threads –(Unknown to kernel) –Extremely lightweight: No system call to needed to change threads.

49. Questions?

50. Things to know? 1. The reason threads are around? 2. Benefits of increased concurrency? 3. Why do we need software controlled "locks" (mutexes) of shared data? 4. How can we avoid potential deadlocks/race conditions. 5. What is meant by producer/consumer thread synchronization/communication using pthreads? 6. Why use a "while" loop around a pthread_cond_wait() call?

51. Things to know? 7. Why should we minimize lock scope (minimize the extent of code within a lock/unlock block)? 8. Do you have any control over thread scheduling?

52. Questions?